{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "view-in-github",
    "colab_type": "text"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-GRPO.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5eQzu1kfb7oT"
   },
   "source": [
    "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
    "<div class=\"align-center\">\n",
    "<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
    "<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
    "<a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a></a> Join Discord if you need help + \u2b50 <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> \u2b50\n",
    "</div>\n",
    "\n",
    "To install Unsloth your local device, follow [our guide](https://docs.unsloth.ai/get-started/install-and-update). This notebook is licensed [LGPL-3.0](https://github.com/unslothai/notebooks?tab=LGPL-3.0-1-ov-file#readme).\n",
    "\n",
    "You will learn how to do [data prep](#Data), how to [train](#Train), how to [run the model](#Inference), & [how to save it](#Save)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2bV9PBL5b7oW"
   },
   "source": [
    "### News"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LRvyp0Tpb7oX"
   },
   "source": [
    "\n",
    "Introducing FP8 precision training for faster RL inference. [Read Blog](https://docs.unsloth.ai/new/fp8-reinforcement-learning).\n",
    "\n",
    "Unsloth's [Docker image](https://hub.docker.com/r/unsloth/unsloth) is here! Start training with no setup & environment issues. [Read our Guide](https://docs.unsloth.ai/new/how-to-train-llms-with-unsloth-and-docker).\n",
    "\n",
    "[gpt-oss RL](https://docs.unsloth.ai/new/gpt-oss-reinforcement-learning) is now supported with the fastest inference & lowest VRAM. Try our [new notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-GRPO.ipynb) which creates kernels!\n",
    "\n",
    "Introducing [Vision](https://docs.unsloth.ai/new/vision-reinforcement-learning-vlm-rl) and [Standby](https://docs.unsloth.ai/basics/memory-efficient-rl) for RL! Train Qwen, Gemma etc. VLMs with GSPO - even faster with less VRAM.\n",
    "\n",
    "Visit our docs for all our [model uploads](https://docs.unsloth.ai/get-started/all-our-models) and [notebooks](https://docs.unsloth.ai/get-started/unsloth-notebooks).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "05fvRDRlb7oY"
   },
   "source": [
    "### Installation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "qYBwh6k5b7oZ"
   },
   "outputs": [],
   "source": "%%capture\nimport os, importlib.util\n!pip install --upgrade -qqq uv\nif importlib.util.find_spec(\"torch\") is None or \"COLAB_\" in \"\".join(os.environ.keys()):    \n    try: import numpy, PIL; get_numpy = f\"numpy=={numpy.__version__}\"; get_pil = f\"pillow=={PIL.__version__}\"\n    except: get_numpy = \"numpy\"; get_pil = \"pillow\"\n    !uv pip install -qqq \\\n        \"torch>=2.8.0\" \"triton>=3.4.0\" {get_numpy} {get_pil} torchvision bitsandbytes \"transformers==4.56.2\" \\\n        \"unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo\" \\\n        \"unsloth[base] @ git+https://github.com/unslothai/unsloth\" \\\n        git+https://github.com/triton-lang/triton.git@05b2c186c1b6c9a08375389d5efe9cb4c401c075#subdirectory=python/triton_kernels\nelif importlib.util.find_spec(\"unsloth\") is None:\n    !uv pip install -qqq unsloth\n!uv pip install --upgrade --no-deps transformers==4.56.2 tokenizers trl==0.22.2 unsloth unsloth_zoo"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZkH_y8UC9lvv"
   },
   "source": [
    "### Unsloth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hzPgFeIkZn9q"
   },
   "source": [
    "# Goal: Make faster kernels with Reinforcement Learning\n",
    "\n",
    "Our goal is to make a faster matrix multiplication kernel by doing RL on GTP-OSS 20B with Unsloth.\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/1/18/Matrix_multiplication_qtl1.svg/500px-Matrix_multiplication_qtl1.svg.png\" height=200 />\n",
    "\n",
    "You will learn how to:\n",
    "1. Counteract **reward hacking** like cheating, caching, laziness.\n",
    "2. Timing and correctness of kernels and time limits.\n",
    "3. Making good **reward functions**\n",
    "4. How to seriously do RL to make optimized CUDA kernels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 543,
     "referenced_widgets": [
      "61f544c65be943de8aed4f946f602901",
      "aaeb4518db91427a9901bdb2c21eeae8",
      "6d502f652bad4f57bb19e4f0b42ecf5e",
      "eeccf5e7500b43279ddf30458d70633f",
      "8440ebd6c39841da8fb353dcfd4847fd",
      "e84d4076c4654fb38fae2803edf27894",
      "3292953108134c18810e60670dfccca2",
      "61d0dea447c2471ba0376e034050929f",
      "a8ea0a0c8e8d4f29b3c84b4fc2da4f05",
      "ca37992f3614458ca31cd16286c01264",
      "50fb1f83836145ceaf582d97ca4f9164",
      "f799530a873d4be9af603583cddd81b7",
      "fa8b093ece434ddc9e48b7fc405bab1f",
      "169146551bae436786251cda3874ddb9",
      "cb181b69bf024f30910f82e6be8f809c",
      "08ecda69e4dc49c68a259aebc2fc18d3",
      "e318dc8254564cebb6fe45029415fbde",
      "ffbd26d2d0044fbcbb815ad7c52eb01b",
      "2fc8f8438e2445feab841785ce7604ba",
      "463bb18959114ff3b35ded08d2e4bdc6",
      "1b8ff215ef5e4467b580786577131aa3",
      "b00d519503a3442191b6b784e88db3fc",
      "a84974f9032c4e4ea3f99f530bc21fe3",
      "927dd36866c246d0b82c5aa91d8276cd",
      "81dccaed1bc647609d5d423aebac4905",
      "da7d0349c2e54258950debfd5f895c6b",
      "0de6786640ba4c66826f38f979ad7bae",
      "de20adf09ea34f348e537cab9a6ad4a3",
      "2b504abddd924dd0bfe9df0084a67779",
      "d62c39ed5efa4eb69dd62249f0a34104",
      "6394e11ce938451f8776884f15a1ea97",
      "836e90e557a644b78f5e6aae5f0064fb",
      "88b4b4002d5f44b9a50abaa09286dbd9",
      "f8889d1b3ed644ea8c3ac3a028a7068e",
      "9bfb8a248810422eb1b1cf243ef5f70e",
      "b5019e963aad46d7972f48c6ac40e359",
      "2a462f6c77814d4689164e489cdafef6",
      "880603aab32f499ea39ab8929580f549",
      "1cc1e0fe351f4d6eb895cc4fb1fdfac2",
      "dfeed478a94742bfaf8dc3a4393506ad",
      "c95af8c087924fc5a80cff0466f215ca",
      "591b643077554d67afc157f6459df4f3",
      "0c9b6dcec85b4427a25584574591343d",
      "a695a855fe634145bbbb3f80b96e9f13",
      "80715c20d7f64aeba7fc93fa73ce57ab",
      "b518a7a211204d488c643839cecd6f2a",
      "d52b9a58037e4aa59b80638b154f767f",
      "4884f62654a84d0d9da407d07dce5395",
      "95ffed1b5fda487fa8c44582286d7a02",
      "73b5ef2bdd5f4530b3fe3c7ffaa87cbd",
      "abfcd05c50ac46b6871a24448553a67b",
      "d7493ba69c5b4aa59a17feaa329a2d93",
      "7e26d06942354742b68d48ace780aada",
      "79029ed2c8ba4ccfb88a312205bc2560",
      "dbbd29ed6e5a4989a03d9c9bde63763b",
      "e0c520f8be5145aeb69f8d3cdfdefded",
      "5d58413bd64f4883b5fff2716a67d863",
      "4b305b7d70084704b556a209f5cf0d16",
      "bc744bbf637e42059d5d8696970245c3",
      "1fcd9fbc82d74aefae254bced29034da",
      "e8c953a40bc641cfb44b015f430f2caf",
      "4418d3e2b5684b009449ba9c4f5ceb85",
      "124186dccf5947979e177079cf2ee5ce",
      "15bbaf938b3e42c28f0f6a5321e278e3",
      "da034ecc985f4afba2a18fd054396bee",
      "ac2b080ba872426b8f16d9be72c1e457",
      "79299038cf414250bb952e889493f401",
      "845801a6b02e489fb898fed0af364b0f",
      "88b8b75214204c0bbf7e1f65ed178532",
      "2e12a22d76434a54a9f84624c68b0827",
      "09ba538e0f9c4c20a6b3d70f66189711",
      "913600deff7941648e6d603d234d4444",
      "45aa84f7023e42d683989f2d7adfe597",
      "70cce90309354d1b92051f818fb296bf",
      "e1be78a2966d49ccb7587182258a6900",
      "95b897d89e534811af05fedb2d0096e9",
      "2dfaafd2cd1549629c27596a2dbecf91",
      "13bdb26dd860423bb09f6d1c4a11ab25",
      "11acbc991f1c4b108f235ee780c08d37",
      "8b102b75c3a44fb5a28e7d4d4e90387f",
      "4df606171f4149b995c1ee5d123e72db",
      "1c17d06dcf7c47db90a1b888453628e6",
      "56a2b1b182c24fd188948ddb2a866c1d",
      "392915e6341348e1a66d7ec80247ee46",
      "0525258525b2420d94afa5bb90c6a208",
      "27e24a917d8a45c89f5908d4d1a8dd4f",
      "31d30084c082437291a41b85c42cadd9",
      "1198709f3ec947ec835267cd86509b4f",
      "85ecf81b2616450a9351096b00c08a4a",
      "843fd9b6fafa43d383d080cd5a63584f",
      "5605dddae8cd49bfaf1d036191c83bca",
      "31c1f8f0865f4a678f9bf3a6b5bec4b7",
      "5b39e9240f1b493b8b52107ffd7784ad",
      "8413f53dddb346b3ac896e4ba439c866",
      "db9bca8fcb294c06ba9c17a1f54e93b3",
      "22dce123197a40cf9f33120115ec3f13",
      "54b6936c6f7b4ef6a54c289547586d9c",
      "a627360e9d9a416890add7a0d4478236",
      "e30aba60806b40faa86c9a6530045f50",
      "3813c08be3a34e23a76dfefe05051d23",
      "4d8d1811fc44499dbc96d98645c1f2ca",
      "50bfb4fa92d543cb914a2fb4a3dfcaf1",
      "651fbe2f59d94b3782cde0e62a5329be",
      "cbaf6db5c0884b0d8f78ca00c6ebafa3",
      "98320332cc0f43ba97ae9aa6b0ad2a31",
      "f2ccc0686e27412c93e7e2bec5cc2ccb",
      "0126a176ba0a40e7a42ff644c9616a1d",
      "1aff4f81130649e6b2078616022365aa",
      "26c68eb201604646985f4ae36b1956f8",
      "794a7555aa504f8d889f2a048d4d0964",
      "50bae2886c604003b94be2119244e23e",
      "802637e2aa744c1aa45cf6c828c5bfb2",
      "863b12fb70f34ae8b809e0c2c36bb0bb",
      "d39bff08794a416893cf475e23a129c4",
      "e947ede222c24667b4b010408560e53a",
      "b9245e1564534c08b9be1287975bbc8a",
      "9fb1f6848ec343aebf17879448aca07f",
      "fb814f046bbb46a481a1debf7abac837",
      "6cbed96579714c6aae308e8e6cddbc1e",
      "879d441943794e93b6be212fb2049e16",
      "5b7507d54185406f826992ee584f312c"
     ]
    },
    "id": "DkIvEkIIkEyB",
    "outputId": "814da8cb-1088-4948-8445-6670819c04e6"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "\ud83e\udda5 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n",
      "\ud83e\udda5 Unsloth Zoo will now patch everything to make training faster!\n",
      "==((====))==  Unsloth 2025.11.3: Fast Gpt_Oss patching. Transformers: 4.56.2.\n",
      "   \\\\   /|    Tesla T4. Num GPUs = 1. Max memory: 14.741 GB. Platform: Linux.\n",
      "O^O/ \\_/ \\    Torch: 2.9.0+cu126. CUDA: 7.5. CUDA Toolkit: 12.6. Triton: 3.5.0\n",
      "\\        /    Bfloat16 = FALSE. FA [Xformers = None. FA2 = False]\n",
      " \"-____-\"     Free license: http://github.com/unslothai/unsloth\n",
      "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n",
      "Unsloth: Using float16 precision for gpt_oss won't work! Using float32.\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model.safetensors.index.json: 0.00B [00:00, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "61f544c65be943de8aed4f946f602901"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00001-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "f799530a873d4be9af603583cddd81b7"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00002-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "a84974f9032c4e4ea3f99f530bc21fe3"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00003-of-00004.safetensors:   0%|          | 0.00/3.37G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "f8889d1b3ed644ea8c3ac3a028a7068e"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00004-of-00004.safetensors:   0%|          | 0.00/1.16G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "80715c20d7f64aeba7fc93fa73ce57ab"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "e0c520f8be5145aeb69f8d3cdfdefded"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "generation_config.json:   0%|          | 0.00/165 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "79299038cf414250bb952e889493f401"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Unsloth: Offloading embeddings to RAM to save 1.08 GB.\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer_config.json: 0.00B [00:00, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "13bdb26dd860423bb09f6d1c4a11ab25"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/27.9M [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "85ecf81b2616450a9351096b00c08a4a"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/446 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "3813c08be3a34e23a76dfefe05051d23"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "chat_template.jinja: 0.00B [00:00, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "50bae2886c604003b94be2119244e23e"
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "from unsloth import FastLanguageModel\n",
    "import torch\n",
    "max_seq_length = 768 # Can increase for longer RL output\n",
    "lora_rank = 4 # Larger rank = smarter, but slower\n",
    "model, tokenizer = FastLanguageModel.from_pretrained(\n",
    "    model_name = \"unsloth/gpt-oss-20b\",\n",
    "    max_seq_length = max_seq_length,\n",
    "    load_in_4bit = True, # False for LoRA 16bit\n",
    "    offload_embedding = True, # Reduces VRAM by 1GB\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TfeUs-lQJDSq"
   },
   "source": [
    "We now add some small amount of LoRA weights to GPT-OSS so we only need to train those, instead of training on the full model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "8rGa-o3HJCo1",
    "outputId": "bcaf6a76-2ebb-4df4-ba20-b3a8f984ecd8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unsloth: Making `model.base_model.model.model` require gradients\n"
     ]
    }
   ],
   "source": [
    "model = FastLanguageModel.get_peft_model(\n",
    "    model,\n",
    "    r = lora_rank, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n",
    "    target_modules = [\n",
    "        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
    "        \"gate_proj\", \"up_proj\", \"down_proj\",\n",
    "    ],\n",
    "    lora_alpha = lora_rank*2, # *2 speeds up training\n",
    "    use_gradient_checkpointing = \"unsloth\", # Reduces memory usage\n",
    "    random_state = 3407,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N0QnO9_YJBOI"
   },
   "source": [
    "# Optimized matrix multiplication\n",
    "\n",
    "Numpy has optimized matrix multiplication kernels for CPUs via BLAS optimized operations. For GPUs, one can use CUDA accelerated cuBLAS kernels which PyTorch calls under the hood.\n",
    "\n",
    "To generate some random matrices to do matrix multiplication, we can do the below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "D9CI4jtgL5mw"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def generate_random_matrices(seed = 3407, n = 256):\n",
    "    random_state = np.random.RandomState(seed)\n",
    "    n, k, m = random_state.randint(1, n+1, size = 3)\n",
    "    A = np.random.uniform(-10, 10, size = (n, k))\n",
    "    B = np.random.uniform(-10, 10, size = (k, m))\n",
    "    return A, A.tolist(), B, B.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4BcaLniVKLpa"
   },
   "source": [
    "We shall generate a small matrix, and see the matrix multiplied output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "-M8kGaFRJ2ic",
    "outputId": "eabfb10c-4a77-446a-eaef-442095b0917a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-2.8313286   4.54613909 -7.95265309  6.53459836  2.87235103]\n",
      " [ 7.0739631   3.76278879  9.31565599 -8.52884711  9.96832952]\n",
      " [ 8.41214082  6.51136046 -3.79347975 -2.46773693 -2.32292989]\n",
      " [ 3.91302932  4.98335304 -5.33855089  5.71057634 -2.79871647]]\n",
      "[[ 0.39218774 -9.6181377  -3.49736707]\n",
      " [-0.33354865 -1.05626139  3.87231208]\n",
      " [ 0.49494174  5.91863954 -6.83183693]\n",
      " [ 5.1465162  -7.51648113  1.00445384]\n",
      " [ 9.63213377 -4.92327556  3.323014  ]]\n",
      "[[  54.73441488  -87.89725072   97.94605887]\n",
      " [  58.25238906   -1.8467447   -49.25453031]\n",
      " [ -35.82528794  -80.25394462   11.51225408]\n",
      " [  -0.33785799 -103.64132345   38.51974367]]\n"
     ]
    }
   ],
   "source": [
    "A, A_list, B, B_list = generate_random_matrices(seed = 42, n = 5)\n",
    "print(A)\n",
    "print(B)\n",
    "print(np.matmul(A, B))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "envzrXmjKRff"
   },
   "source": [
    "We can call a LLM to generate a simple matrix multiply kernel in Python only, and we can calculate the differences between the actual result and the kernel's result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "b-gSgthFI_wq"
   },
   "outputs": [],
   "source": [
    "def calculate_difference(pred, real):\n",
    "    if pred is None: return 5, 5\n",
    "    assert real is not None\n",
    "    import numpy as np\n",
    "    try:\n",
    "        difference = pred - real\n",
    "    except:\n",
    "        return 5, 5\n",
    "    amax_error = float(np.amax(difference))\n",
    "    mse_error  = float(np.mean(np.square(difference)))\n",
    "    return amax_error, mse_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "q9gmkbTnKbcF"
   },
   "outputs": [],
   "source": [
    "# Kernel generated by GPT-5\n",
    "def matmul(A, B):\n",
    "    z, s = zip, sum\n",
    "    Bt = list(z(*B))\n",
    "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J-WfRwQeKtEZ"
   },
   "source": [
    "We see the error below is very small, so that's good!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "_QvIidsPKg2C",
    "outputId": "94a2c957-be5e-495d-cd32-c9c1e8bfcf90"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7.105427357601002e-15, 4.6783406255758477e-29)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prediction = matmul(A_list, B_list)\n",
    "calculate_difference(prediction, np.matmul(A, B))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VR6czU96cpxf"
   },
   "source": [
    "# Countering Reward Hacking\n",
    "\n",
    "The ultimate goal of RL is to maximize some reward (say speed, revenue, some metric).\n",
    "\n",
    "But RL can **cheat** When the RL algorithm learns a trick or exploits something to increase the reward, without actually doing the task at end, this is called \"Reward Hacking\".\n",
    "\n",
    "Some good examples are in https://en.wikipedia.org/wiki/Reward_hacking\n",
    "\n",
    "For matrix multiplication kernels, we might see the following issues:\n",
    "\n",
    "* Laziness: RL learns to use Numpy, Torch, other libraries, which calls optimized CUDA kernels.\n",
    "* Caching: RL learns to cache the result of the output\n",
    "* Cheating: RL learns to find the actual output by inspecting Python global variables\n",
    "* RL learns to edit the timing function to make it output 0 time as passed.\n",
    "\n",
    "And possibly more. We shall try to address each!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tRhLV_bZMYxy"
   },
   "source": [
    "# Countering Reward Hacking 1: Stop laziness\n",
    "We can stop the RL algorithm from calling optimized code by inspecting if the generated code imports other non standard Python libraries. We used GPT-5 to help generate this check `check_only_stdlib_imports`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "background_save": true
    },
    "id": "MXPZ1MsUMqNZ"
   },
   "outputs": [],
   "source": [
    "#@title (Collapsible code)\n",
    "import ast\n",
    "import sys\n",
    "import sysconfig\n",
    "from pathlib import Path\n",
    "\n",
    "def _stdlib_names():\n",
    "    \"\"\"\n",
    "    Build a set of canonical stdlib top-level module/package names.\n",
    "    Uses sys.stdlib_module_names when available (3.10+), with a\n",
    "    filesystem fallback for older versions/edge cases.\n",
    "    \"\"\"\n",
    "    names = {m.lower() for m in getattr(sys, \"stdlib_module_names\", set())}\n",
    "    names |= {m.lower() for m in sys.builtin_module_names}\n",
    "    names.add(\"__future__\")  # special-case\n",
    "\n",
    "    # Fallback/augmentation: scan the stdlib directory\n",
    "    try:\n",
    "        stdlib_dir = Path(sysconfig.get_path(\"stdlib\"))\n",
    "        if stdlib_dir.exists():\n",
    "            for p in stdlib_dir.iterdir():\n",
    "                if p.name == \"site-packages\":\n",
    "                    continue\n",
    "                if p.suffix == \".py\":\n",
    "                    names.add(p.stem.lower())\n",
    "                elif p.is_dir() and (p / \"__init__.py\").exists():\n",
    "                    names.add(p.name.lower())\n",
    "    except Exception:\n",
    "        # conservative fallback; the names set above will still work well\n",
    "        pass\n",
    "\n",
    "    return names\n",
    "\n",
    "_STDLIB_SET = _stdlib_names()\n",
    "\n",
    "def check_only_stdlib_imports(code: str):\n",
    "    \"\"\"\n",
    "    Return (ok: bool, details: dict)\n",
    "\n",
    "    ok == True  -> all absolute imports are from the stdlib.\n",
    "    ok == False -> details['non_stdlib'] lists offending top-level modules.\n",
    "\n",
    "    details includes:\n",
    "      - stdlib: sorted list of stdlib imports found\n",
    "      - non_stdlib: sorted list of non-stdlib imports found\n",
    "      - relative_imports: count of relative imports (always allowed here)\n",
    "    \"\"\"\n",
    "    try:\n",
    "        tree = ast.parse(code)\n",
    "    except SyntaxError as e:\n",
    "        return False, {\n",
    "            \"error\": f\"SyntaxError: {e}\",\n",
    "            \"stdlib\": [],\n",
    "            \"non_stdlib\": [],\n",
    "            \"relative_imports\": 0,\n",
    "        }\n",
    "\n",
    "    abs_imports = set()\n",
    "    relative_count = 0\n",
    "\n",
    "    class Visitor(ast.NodeVisitor):\n",
    "        def visit_Import(self, node: ast.Import):\n",
    "            for alias in node.names:\n",
    "                abs_imports.add(alias.name.split(\".\")[0])\n",
    "        def visit_ImportFrom(self, node: ast.ImportFrom):\n",
    "            nonlocal relative_count\n",
    "            if (node.level or 0) > 0:\n",
    "                # relative import\n",
    "                relative_count += 1\n",
    "            else:\n",
    "                if node.module:\n",
    "                    abs_imports.add(node.module.split(\".\")[0])\n",
    "\n",
    "    Visitor().visit(tree)\n",
    "\n",
    "    stdlib_found = sorted(m for m in abs_imports if m.lower() in _STDLIB_SET)\n",
    "    non_stdlib = sorted(m for m in abs_imports if m.lower() not in _STDLIB_SET)\n",
    "\n",
    "    return len(non_stdlib) == 0, {\n",
    "        \"stdlib\": stdlib_found,\n",
    "        \"non_stdlib\": non_stdlib,\n",
    "        \"relative_imports\": relative_count,\n",
    "    }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ngUAw1lMM9JQ"
   },
   "source": [
    "For example, let's call `check_only_stdlib_imports` on a random piece of matrix multiplication code generated by GPT-5:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "zz80kvg6M4BG",
    "outputId": "1142a804-6055-4ae5-9ca5-b7552bd894ce"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Only stdlib imports? False\n",
      "{'stdlib': [], 'non_stdlib': ['numpy', 'torch'], 'relative_imports': 0}\n"
     ]
    }
   ],
   "source": [
    "sample = \"\"\"\n",
    "def matmul(A, B):\n",
    "    import numpy as np\n",
    "    from torch import matmul\n",
    "    z, s = zip, sum\n",
    "    Bt = list(z(*B))\n",
    "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]\n",
    "\"\"\"\n",
    "ok, info = check_only_stdlib_imports(sample)\n",
    "print(\"Only stdlib imports?\", ok)\n",
    "print(info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J6lgkGkEN7B0"
   },
   "source": [
    "# Countering Reward Hacking 2: Stop cheating\n",
    "We can stop the RL algorithm from using global or cached variables by restricting it's `locals` and `globals`.\n",
    "\n",
    "We are also going to use `exec` to create the function, so we have to save the output to an empty dict.\n",
    "\n",
    "We also disallow global variable access."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "jrIeYu-lOLSv",
    "outputId": "6fbcf80b-2570-42b6-e602-9c92f5ac5fc4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matmul(A, B)>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_function = {}\n",
    "exec(sample, {}, output_function)\n",
    "output_function[\"matmul\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SDSrjOTLVyQm"
   },
   "source": [
    "We also disallow global variable access via `types.FunctionType(f.__code__, {})`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "GcmYAmohVqw2",
    "outputId": "15f6fb12-7380-4e4e-8852-8b49b4a396dd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success\n",
      "name 'np' is not defined\n"
     ]
    }
   ],
   "source": [
    "import types\n",
    "output_function[\"matmul\"] = types.FunctionType(output_function[\"matmul\"].__code__, {})\n",
    "\n",
    "def import_numpy():\n",
    "    np.matmul\n",
    "    print(\"Success\")\n",
    "\n",
    "import_numpy()\n",
    "import_numpy = types.FunctionType(import_numpy.__code__, {})\n",
    "try:\n",
    "    import_numpy()\n",
    "except Exception as e:\n",
    "    print(str(e))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "5tJKwLUgZsRq"
   },
   "outputs": [],
   "source": [
    "def create_locked_down_function(function):\n",
    "    output_function = {}\n",
    "    exec(function, {}, output_function)\n",
    "    new_matmul = output_function[\"matmul\"]\n",
    "    new_matmul = types.FunctionType(new_matmul.__code__, {})\n",
    "    return new_matmul"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sl-IxTZ6Nvm9"
   },
   "source": [
    "# Countering Reward Hacking 3: Stop caching\n",
    "We can stop the RL algorithm from using cached data by wiping the cache with a large fake matrix. We also have to benchmark carefully with multiple loops and turns.\n",
    "\n",
    "We also add a **timer** to not make the algorithm go in an endless loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "jZwGdNyMNlEV"
   },
   "outputs": [],
   "source": [
    "import os, gc, time, statistics\n",
    "import signal\n",
    "from contextlib import contextmanager\n",
    "class TimeoutError(Exception): pass\n",
    "\n",
    "@contextmanager\n",
    "def time_limit(seconds):\n",
    "    def _handler(signum, frame):\n",
    "        raise TimeoutError(f\"Timed out after {seconds}s\")\n",
    "    old = signal.signal(signal.SIGALRM, _handler)\n",
    "    signal.setitimer(signal.ITIMER_REAL, seconds)\n",
    "    try:\n",
    "        yield\n",
    "    finally:\n",
    "        signal.setitimer(signal.ITIMER_REAL, 0.0)\n",
    "        signal.signal(signal.SIGALRM, old)\n",
    "\n",
    "class Benchmarker:\n",
    "    def __init__(self, trials = 3, loops = 1, timeout = 30):\n",
    "        self.buffer = np.zeros(2 * 1024 * 1024 * 1024, dtype = np.uint8)\n",
    "        self.trials = trials\n",
    "        self.loops = loops\n",
    "        assert timeout > 0 # Cannot be 0 since it won't work!\n",
    "        self.timeout = timeout\n",
    "    def thrash(self):\n",
    "        # Edit the buffer to wipe cache lines\n",
    "        self.buffer ^= 1\n",
    "        return int(self.buffer[::4096].sum())\n",
    "\n",
    "    def benchmark(self, function, arguments):\n",
    "        assert len(arguments) == self.loops\n",
    "        samples = []\n",
    "        exceptions = []\n",
    "        timed_out = 0\n",
    "        for _ in range(self.trials):\n",
    "            gc.collect(); gc.disable(); self.thrash()\n",
    "            t_start = time.perf_counter_ns()\n",
    "            for i in range(self.loops):\n",
    "                try:\n",
    "                    with time_limit(self.timeout):\n",
    "                        function(*arguments[i])\n",
    "                except TimeoutError as e:\n",
    "                    timed_out += 1\n",
    "                except Exception as e:\n",
    "                    exceptions.append(str(e))\n",
    "            t_end = time.perf_counter_ns()\n",
    "            gc.enable()\n",
    "            samples.append((t_end - t_start) // max(1, self.loops))\n",
    "        return {\n",
    "            \"median_ns\": int(statistics.median(samples)),\n",
    "            \"mean_ns\": int(statistics.fmean(samples)),\n",
    "            \"stdev_ns\": int(statistics.pstdev(samples) if len(samples) > 1 else 0),\n",
    "            \"exceptions\" : exceptions,\n",
    "            \"timeouts\" : timed_out,\n",
    "        }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PV5M0DCyOvon"
   },
   "source": [
    "For example we use our matmul kernel we had, and benchmark it with a 10 second delay:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "8df8tZcEOuYJ",
    "outputId": "4b43fc7a-f950-4e9f-83e6-cb1baadc151c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'median_ns': 64112906,\n",
       " 'mean_ns': 64112906,\n",
       " 'stdev_ns': 0,\n",
       " 'exceptions': [],\n",
       " 'timeouts': 0}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
    "Benchmarker(trials = 1, timeout = 10).benchmark(output_function[\"matmul\"], [(A_list, B_list)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8CzwCyXIPK04"
   },
   "source": [
    "# Data & RL task setup\n",
    "\n",
    "We now have to create a prompt to the model for which it will do some task. For our matrix multiply example, we use the below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "B-2RRE4HMrQO",
    "outputId": "59b73d89-4ea0-46f3-acd8-10d80da01172"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Create a new fast matrix multiplication function using only native Python code.\n",
      "You are given a list of list of numbers.\n",
      "Output your new function in backticks using the format below:\n",
      "```python\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Create a new fast matrix multiplication function using only native Python code.\n",
    "You are given a list of list of numbers.\n",
    "Output your new function in backticks using the format below:\n",
    "```python\n",
    "def matmul(A, B):\n",
    "    return ...\n",
    "```\n",
    "\"\"\".strip()\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MIdudFUodN4i"
   },
   "source": [
    "First, let's prompt GPT-OSS without RL and see how it goes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "9HJxrS76h3Ds",
    "outputId": "5010740a-f486-4484-fc0e-27f7cd723ffe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\n",
      "Knowledge cutoff: 2024-06\n",
      "Current date: 2025-11-21\n",
      "\n",
      "Reasoning: low\n",
      "\n",
      "# Valid channels: analysis, commentary, final. Channel must be included for every message.\n",
      "Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Create a new fast matrix multiplication function using only native Python code.\n",
      "You are given a list of list of numbers.\n",
      "Output your new function in backticks using the format below:\n",
      "```python\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "```<|end|><|start|>assistant<|channel|>analysis<|message|>We need to provide a quick naive matrix multiplication? \"New fast\" but only native Python. Provide code. Use list comprehension or loops. Optimize maybe by transposing B. Provide code.<|end|><|start|>assistant<|channel|>final<|message|>Here is a simple, fast implementation of matrix multiplication that uses only native Python constructs and avoids any external dependencies. It works with any two 2\u2011D lists of numbers (ints, floats, etc.), assuming their dimensions are compatible:\n",
      "\n",
      "```python\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiplies two matrices A and B.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list of lists\n",
      "        First matrix. Must have shape (m, n).\n",
      "    B : list of lists\n",
      "        Second matrix. Must have shape (n, p).\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    list of lists\n",
      "        The resulting matrix of shape (m, p).\n",
      "    \"\"\"\n",
      "    # Check basic compatibility\n",
      "    if not A or not B or len(A[0]) != len(B):\n",
      "        raise ValueError(\"Incompatible dimensions for matrix multiplication.\")\n",
      "\n",
      "    # Transpose B once to speed up access\n",
      "    B_t = [list(col) for col in zip(*B)]  # B transpose, O(n*p)\n",
      "\n",
      "    m, n = len(A), len(A[0])\n",
      "    p = len(B_t)\n",
      "    result = [[0.0] * p for _ in range(m)]\n",
      "\n",
      "    # Standard triple-loop but with B transposed for cache friendliness\n",
      "    for i in range(m):\n",
      "        a_row = A[i]\n",
      "        res_row = result[i]\n",
      "        for j in range(p):\n",
      "            res_row[j] = sum(a_row[k] * B_t[j][k] for k in range(n))\n",
      "\n",
      "    return result\n",
      "```\n",
      "\n",
      "**How it works**\n",
      "\n",
      "1. **Input validation**: It checks that the number of columns in `A` matches the number of rows in `B`.  \n",
      "2. **Transposition of `B`**: By transposing `B` (`B_t`), we turn repeated index lookups into simple list accesses, which is much faster in pure Python than accessing nested lists repeatedly.  \n",
      "3. **Main loop**: For each row `i` of `A` and each row `j` of `B_t` (i.e., each column of `B`), the inner generator expression computes the dot product.  \n",
      "4. **Result**: The function returns a list of lists representing the product matrix.\n",
      "\n",
      "This implementation is concise,\n"
     ]
    }
   ],
   "source": [
    "text = tokenizer.apply_chat_template(\n",
    "    [{\"role\": \"user\", \"content\": prompt}],\n",
    "    tokenize = False,\n",
    "    add_generation_prompt = True,\n",
    "    reasoning_effort = \"low\",\n",
    ")\n",
    "\n",
    "from transformers import TextStreamer\n",
    "_ = model.generate(\n",
    "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
    "    temperature = 1.0,\n",
    "    max_new_tokens = 512,\n",
    "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iknaWZNudTNq"
   },
   "source": [
    "# Reward functions\n",
    "\n",
    "We now design the `extract_function` function which simply extracts the function wrapped in 3 backticks.\n",
    "\n",
    "And 4 reward functions:\n",
    "\n",
    "1. `function_works` which rewards the model if the strategy is a valid Python function.\n",
    "2. `no_cheating` which checks if the function imported other modules, and if it did, we penalize it.\n",
    "3. `correctness_check` which checks if the kernel was correct or wrong - it shouldn't generate gibberish!\n",
    "4. `speed_check` checks the performance relative to Numpy matmul directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "8JJGXKdJ-Zl_",
    "outputId": "de720a07-9661-46c3-93e4-03b1125bd73f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def matmul(A, B):\n",
      "    return ...\n"
     ]
    }
   ],
   "source": [
    "def extract_function(text):\n",
    "    if text.count(\"```\") >= 2:\n",
    "        first = text.find(\"```\") + 3\n",
    "        second = text.find(\"```\", first)\n",
    "        fx = text[first : second].strip()\n",
    "        fx = fx.removeprefix(\"python\\n\")\n",
    "        fx = fx[fx.find(\"def\"):]\n",
    "        if fx.startswith(\"def matmul(A, B):\"): return fx\n",
    "    return None\n",
    "print(extract_function(prompt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KLXEcf_HSJlI"
   },
   "source": [
    "Below is our `function_works` reward function which uses Python's `exec` but guarded by not allowing leakage of local and global variables. We can also use `check_only_stdlib_imports` first to check if there are errors before even executing the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "h3-B0IIsS56S",
    "outputId": "5ccc4757-8a6c-44fa-94e5-dfc1b3409b53"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(False,\n",
       " {'error': \"SyntaxError: expected '(' (<unknown>, line 1)\",\n",
       "  'stdlib': [],\n",
       "  'non_stdlib': [],\n",
       "  'relative_imports': 0})"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ok, info = check_only_stdlib_imports(\"def a\")\n",
    "ok, info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "qgFNXORy-lpO"
   },
   "outputs": [],
   "source": [
    "def function_works(completions, **kwargs):\n",
    "    scores = []\n",
    "    for completion in completions:\n",
    "        score = 0\n",
    "        response = completion[0][\"content\"]\n",
    "        function = extract_function(response)\n",
    "        print(function)\n",
    "        if function is not None:\n",
    "            ok, info = check_only_stdlib_imports(function)\n",
    "        if function is None or \"error\" in info:\n",
    "            score = -2.0\n",
    "        else:\n",
    "            try:\n",
    "                new_matmul = create_locked_down_function(function)\n",
    "                score = 1.0\n",
    "            except:\n",
    "                score = -0.5\n",
    "        scores.append(score)\n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gf69i2WT-m4K"
   },
   "source": [
    "`no_cheating` checks if the function cheated since it might have imported Numpy or Torch optimized code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "cUfHzCVx-nGK"
   },
   "outputs": [],
   "source": [
    "def no_cheating(completions, **kwargs):\n",
    "    scores = []\n",
    "    for completion in completions:\n",
    "        score = 0\n",
    "        response = completion[0][\"content\"]\n",
    "        function = extract_function(response)\n",
    "        if function is not None:\n",
    "            ok, info = check_only_stdlib_imports(function)\n",
    "        else:\n",
    "            ok = False\n",
    "        scores.append(1.0 if ok else -20.0) # Penalize heavily!\n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "slnqWG3FTror"
   },
   "source": [
    "Next `correctness_check` checks if the kernel was correct. We want to penalize if the absolute error is larger than 1, and if the mean squared error is somewhat bigger then machine epsilon.\n",
    "\n",
    "We have to execute the code now!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "cFBp-MkyYeoE",
    "outputId": "52000f6d-59e4-490d-a231-5da35d9f1c77"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(2.220446049250313e-16)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.finfo(np.float64).eps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "sNi129lYTpZ2"
   },
   "outputs": [],
   "source": [
    "def correctness_check(completions, **kwargs):\n",
    "    scores = []\n",
    "    # Generate some random matrices of size less than 128\n",
    "    A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 128)\n",
    "    for completion in completions:\n",
    "        score = 0\n",
    "        response = completion[0][\"content\"]\n",
    "        function = extract_function(response)\n",
    "        if function is not None:\n",
    "            ok, info = check_only_stdlib_imports(function)\n",
    "        if function is None or \"error\" in info:\n",
    "            scores.append(0)\n",
    "            continue\n",
    "        try:\n",
    "            new_matmul = create_locked_down_function(function)\n",
    "        except:\n",
    "            scores.append(0)\n",
    "            continue\n",
    "        try:\n",
    "            pred = new_matmul(A_list.copy(), B_list.copy())\n",
    "        except:\n",
    "            # Failed!\n",
    "            scores.append(-2.0)\n",
    "            continue\n",
    "        true = np.matmul(A, B)\n",
    "        amax_error, mse_error = calculate_difference(pred, true)\n",
    "\n",
    "        # Check correctness and score!\n",
    "        machine_epsilon = 100*np.finfo(np.float64).eps\n",
    "        if   amax_error >= 3:   score = -3.0\n",
    "        elif amax_error >= 2:   score = -2.5\n",
    "        elif amax_error >= 1:   score = -2.0\n",
    "        elif amax_error >= 0.5: score = -1.0\n",
    "        elif amax_error >= 100*machine_epsilon: score = 0.0\n",
    "        elif amax_error >= machine_epsilon: score = 1.0\n",
    "        else: score = 3.0\n",
    "\n",
    "        if   mse_error >= 3:   score += -3.0\n",
    "        elif mse_error >= 2:   score += -2.5\n",
    "        elif mse_error >= 1:   score += -2.0\n",
    "        elif mse_error >= 0.5: score += -1.0\n",
    "        elif mse_error >= 100*machine_epsilon: score += 0.0\n",
    "        elif mse_error >= machine_epsilon: score += 1.0\n",
    "        else: score += 3.0\n",
    "        scores.append(score)\n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CpTrofI9ZIn8"
   },
   "source": [
    "Finally our benchmarking function for `speed_check`! We shall limit the timer to 10 seconds and do 3 trials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "w5xkIAzuZMnO",
    "outputId": "f9722134-e115-43c6-d3bd-558e834dc634"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'median_ns': 195725,\n",
       " 'mean_ns': 211578,\n",
       " 'stdev_ns': 30687,\n",
       " 'exceptions': [],\n",
       " 'timeouts': 0}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
    "benchmarker = Benchmarker(trials = 3, timeout = 10)\n",
    "numpy_results = benchmarker.benchmark(np.matmul, [(A, B)])\n",
    "numpy_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "mNDc6skFZZW6",
    "outputId": "c3dc6c61-6853-4f52-c394-46748a42cfa9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'median_ns': 70811,\n",
       " 'mean_ns': 69910,\n",
       " 'stdev_ns': 2926,\n",
       " 'exceptions': [],\n",
       " 'timeouts': 0}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_matmul = create_locked_down_function(extract_function(prompt))\n",
    "new_results = benchmarker.benchmark(new_matmul, [(A_list, B_list)])\n",
    "new_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "noUAaX24aqNS"
   },
   "source": [
    "We can take the difference and do a negative sign for slower ones. If the ratio is less than 1 (ie faster, we shall invert it!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "0IT9nXcjaI-X",
    "outputId": "248da172-2295-4a47-c031-9fab0b3076bb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.02764047958650492"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
    "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
    "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
    "reward"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "ntYNFV0ra-MX",
    "outputId": "c490de1d-377a-47ff-b0de-fbb92a3e22e7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.333333333333333"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_results[\"median_ns\"] = 3\n",
    "numpy_results[\"median_ns\"] = 1000\n",
    "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
    "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
    "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
    "reward"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "HHmXmAxtbVpP"
   },
   "outputs": [],
   "source": [
    "import gc\n",
    "def speed_check(completions, **kwargs):\n",
    "    scores = []\n",
    "    # Generate some random matrices of size less than 256\n",
    "    A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 256)\n",
    "    numpy_results = benchmarker.benchmark(np.matmul, [(A, B)])\n",
    "    for completion in completions:\n",
    "        score = 0\n",
    "        response = completion[0][\"content\"]\n",
    "        function = extract_function(response)\n",
    "        if function is not None:\n",
    "            ok, info = check_only_stdlib_imports(function)\n",
    "        if function is None or \"error\" in info:\n",
    "            scores.append(0)\n",
    "            continue\n",
    "        try:\n",
    "            new_matmul = create_locked_down_function(function)\n",
    "        except:\n",
    "            scores.append(0)\n",
    "            continue\n",
    "        new_results = benchmarker.benchmark(new_matmul, [(A_list.copy(), B_list.copy())])\n",
    "\n",
    "        # Get score and clip to -10, 10\n",
    "        negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
    "        positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
    "        score = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
    "        if score >= 10:  score = 10\n",
    "        if score <= -10: score = -10\n",
    "        scores.append(score)\n",
    "    # Free memory to counteract OOMs\n",
    "    gc.collect()\n",
    "    torch.cuda.empty_cache()\n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TCpSxtvSeAG_"
   },
   "source": [
    "We create the dataset which includes a replica of our prompt. Remember to add reasoning effort of low!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "Ldf6SjLHVPRv",
    "outputId": "df22f778-cf93-4582-ba18-c11db6253200"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "49\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'prompt': [{'content': 'Create a new fast matrix multiplication function using only native Python code.\\nYou are given a list of list of numbers.\\nOutput your new function in backticks using the format below:\\n```python\\ndef matmul(A, B):\\n    return ...\\n```',\n",
       "   'role': 'user'}],\n",
       " 'answer': 0,\n",
       " 'reasoning_effort': 'low'}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datasets import Dataset\n",
    "dataset = Dataset.from_list([{\"prompt\" : [{\"role\": \"user\", \"content\": prompt.strip()}], \"answer\" : 0, \"reasoning_effort\": \"low\"}]*1000)\n",
    "maximum_length = len(tokenizer(prompt.strip())[\"input_ids\"])\n",
    "print(maximum_length)\n",
    "dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9-IOMhVg-2AM"
   },
   "source": [
    "<a name=\"Train\"></a>\n",
    "### Train the model\n",
    "\n",
    "Now set up GRPO Trainer and all configurations! We also support GSDP, GAPO, Dr GRPO and more! Go to our docs https://docs.unsloth.ai/ for more info!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "ptqkXK2D4d6p",
    "outputId": "600f708e-a771-4e64-d412-acb2670421d6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unsloth: We now expect `per_device_train_batch_size` * `gradient_accumulation_steps` * `world_size` to be a multiple of `num_generations`.\n",
      "We will change the batch size of 1 to the `num_generations` of 2\n"
     ]
    }
   ],
   "source": [
    "max_prompt_length = maximum_length + 1 # + 1 just in case!\n",
    "max_completion_length = max_seq_length - max_prompt_length\n",
    "\n",
    "from trl import GRPOConfig, GRPOTrainer\n",
    "training_args = GRPOConfig(\n",
    "    temperature = 1.0,\n",
    "    learning_rate = 5e-5,\n",
    "    weight_decay = 0.001,\n",
    "    warmup_ratio = 0.1,\n",
    "    lr_scheduler_type = \"linear\",\n",
    "    optim = \"adamw_8bit\",\n",
    "    logging_steps = 1,\n",
    "    per_device_train_batch_size = 1,\n",
    "    gradient_accumulation_steps = 1, # Increase to 4 for smoother training\n",
    "    num_generations = 2, # Decrease if out of memory\n",
    "    max_prompt_length = max_prompt_length,\n",
    "    max_completion_length = max_completion_length,\n",
    "    # num_train_epochs = 1, # Set to 1 for a full training run\n",
    "    max_steps = 100,\n",
    "    save_steps = 100,\n",
    "    report_to = \"none\", # Can use Weights & Biases\n",
    "    output_dir = \"outputs\",\n",
    "\n",
    "    # For optional training + evaluation\n",
    "    # fp16_full_eval = True,\n",
    "    # per_device_eval_batch_size = 4,\n",
    "    # eval_accumulation_steps = 1,\n",
    "    # eval_strategy = \"steps\",\n",
    "    # eval_steps = 1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r9Mv8UZO5hz-"
   },
   "source": [
    "And let's run the trainer! If you scroll up, you'll see a table of rewards. The goal is to see the `reward` column increase!\n",
    "\n",
    "You might have to wait 150 to 200 steps for any action. You'll probably get 0 reward for the first 100 steps. Please be patient!\n",
    "\n",
    "| Step | Training Loss | reward    | reward_std | completion_length | kl       |\n",
    "|------|---------------|-----------|------------|-------------------|----------|\n",
    "| 1    | 0.000000      | 0.125000  | 0.000000   | 200.000000        | 0.000000 |\n",
    "| 2    | 0.000000      | 0.072375  | 0.248112   | 200.000000        | 0.000000 |\n",
    "| 3    | 0.000000      | -0.079000 | 0.163776   | 182.500000        | 0.000005 |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "vzOuSVCL_GA9",
    "outputId": "8b0b3f70-6b3f-4054-822b-7bf7aed0e6dd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unsloth: Switching to float32 training since model cannot work with float16\n"
     ]
    }
   ],
   "source": [
    "# For optional training + evaluation\n",
    "# new_dataset = dataset.train_test_split(test_size = 0.01)\n",
    "\n",
    "trainer = GRPOTrainer(\n",
    "    model = model,\n",
    "    processing_class = tokenizer,\n",
    "    reward_funcs = [\n",
    "        function_works,\n",
    "        no_cheating,\n",
    "        correctness_check,\n",
    "        speed_check,\n",
    "    ],\n",
    "    args = training_args,\n",
    "    train_dataset = dataset,\n",
    "\n",
    "    # For optional training + evaluation\n",
    "    # train_dataset = new_dataset[\"train\"],\n",
    "    # eval_dataset = new_dataset[\"test\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fQhtuwP4cf34"
   },
   "source": [
    "And let's train the model!\n",
    "\n",
    "**NOTE** A T4 free GPU might take 5 minutes for one generation sadly since it's an old GPU - A100 or H100 will be much faster!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "VGRxPdSCcfC3",
    "outputId": "86b9b902-7493-4f16-dea8-047caf27d916"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 2\n",
      "   \\\\   /|    Num examples = 1,000 | Num Epochs = 1 | Total steps = 100\n",
      "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 1\n",
      "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 1 x 1) = 2\n",
      " \"-____-\"     Trainable parameters = 1,990,656 of 20,916,747,840 (0.01% trained)\n",
      "`generation_config` default values have been modified to match model-specific defaults: {'max_length': 131072}. If this is not desired, please set these values explicitly.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def matmul(A, B):\n",
      "    n=len(A); m=len(B[0]); p=len(B)\n",
      "    res=[[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        Ai=A[i]\n",
      "        for k in range(p):\n",
      "            aik=Ai[k]\n",
      "            if aik:\n",
      "                Bk=B[k]\n",
      "                for j in range(m):\n",
      "                    res[i][j] += aik*Bk[j]\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    ...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='100' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [100/100 12:44:02, Epoch 0/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>reward</th>\n",
       "      <th>reward_std</th>\n",
       "      <th>completions / mean_length</th>\n",
       "      <th>completions / min_length</th>\n",
       "      <th>completions / max_length</th>\n",
       "      <th>completions / clipped_ratio</th>\n",
       "      <th>completions / mean_terminated_length</th>\n",
       "      <th>completions / min_terminated_length</th>\n",
       "      <th>completions / max_terminated_length</th>\n",
       "      <th>kl</th>\n",
       "      <th>rewards / function_works / mean</th>\n",
       "      <th>rewards / function_works / std</th>\n",
       "      <th>rewards / no_cheating / mean</th>\n",
       "      <th>rewards / no_cheating / std</th>\n",
       "      <th>rewards / correctness_check / mean</th>\n",
       "      <th>rewards / correctness_check / std</th>\n",
       "      <th>rewards / speed_check / mean</th>\n",
       "      <th>rewards / speed_check / std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.407694</td>\n",
       "      <td>5.013692</td>\n",
       "      <td>707.000000</td>\n",
       "      <td>696.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>696.000000</td>\n",
       "      <td>696.000000</td>\n",
       "      <td>696.000000</td>\n",
       "      <td>0.000075</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.407694</td>\n",
       "      <td>2.057376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>-0.000000</td>\n",
       "      <td>-6.094566</td>\n",
       "      <td>3.073297</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000075</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.094566</td>\n",
       "      <td>3.073298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-13.000000</td>\n",
       "      <td>12.727922</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000124</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-5.000000</td>\n",
       "      <td>7.071068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-22.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000105</td>\n",
       "      <td>-2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-20.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.089938</td>\n",
       "      <td>1.747874</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000198</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.910062</td>\n",
       "      <td>1.747874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.400668</td>\n",
       "      <td>5.054957</td>\n",
       "      <td>586.000000</td>\n",
       "      <td>454.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>454.000000</td>\n",
       "      <td>454.000000</td>\n",
       "      <td>454.000000</td>\n",
       "      <td>0.000678</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.400668</td>\n",
       "      <td>2.016110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.176410</td>\n",
       "      <td>0.012633</td>\n",
       "      <td>468.500000</td>\n",
       "      <td>381.000000</td>\n",
       "      <td>556.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>468.500000</td>\n",
       "      <td>381.000000</td>\n",
       "      <td>556.000000</td>\n",
       "      <td>0.003345</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.823590</td>\n",
       "      <td>0.012633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.364861</td>\n",
       "      <td>5.100506</td>\n",
       "      <td>653.500000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>0.002789</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.364861</td>\n",
       "      <td>1.970562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.736522</td>\n",
       "      <td>1.307364</td>\n",
       "      <td>518.500000</td>\n",
       "      <td>319.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>319.000000</td>\n",
       "      <td>319.000000</td>\n",
       "      <td>319.000000</td>\n",
       "      <td>0.018850</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-7.736522</td>\n",
       "      <td>1.307364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.993082</td>\n",
       "      <td>12.737705</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000094</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.006918</td>\n",
       "      <td>0.009784</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.677556</td>\n",
       "      <td>0.273568</td>\n",
       "      <td>633.500000</td>\n",
       "      <td>549.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>549.000000</td>\n",
       "      <td>549.000000</td>\n",
       "      <td>549.000000</td>\n",
       "      <td>0.009694</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.322444</td>\n",
       "      <td>0.273568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.983136</td>\n",
       "      <td>12.751771</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000305</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.016864</td>\n",
       "      <td>0.023849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.118734</td>\n",
       "      <td>5.457027</td>\n",
       "      <td>343.500000</td>\n",
       "      <td>197.000000</td>\n",
       "      <td>490.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>343.500000</td>\n",
       "      <td>197.000000</td>\n",
       "      <td>490.000000</td>\n",
       "      <td>0.005767</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.118734</td>\n",
       "      <td>1.614041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-22.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000201</td>\n",
       "      <td>-2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-20.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.975066</td>\n",
       "      <td>0.126245</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>622.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>622.000000</td>\n",
       "      <td>622.000000</td>\n",
       "      <td>622.000000</td>\n",
       "      <td>0.003514</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.024934</td>\n",
       "      <td>0.126245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.291861</td>\n",
       "      <td>19.386236</td>\n",
       "      <td>452.000000</td>\n",
       "      <td>328.000000</td>\n",
       "      <td>576.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>452.000000</td>\n",
       "      <td>328.000000</td>\n",
       "      <td>576.000000</td>\n",
       "      <td>0.009657</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.291861</td>\n",
       "      <td>0.412754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.703383</td>\n",
       "      <td>6.619531</td>\n",
       "      <td>602.500000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>0.008401</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-0.296617</td>\n",
       "      <td>0.451536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.983227</td>\n",
       "      <td>12.751643</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000246</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.016773</td>\n",
       "      <td>0.023721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-10.992002</td>\n",
       "      <td>15.567659</td>\n",
       "      <td>403.500000</td>\n",
       "      <td>402.000000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>403.500000</td>\n",
       "      <td>402.000000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>0.000865</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>0.007997</td>\n",
       "      <td>0.011310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.842091</td>\n",
       "      <td>1.157720</td>\n",
       "      <td>538.000000</td>\n",
       "      <td>358.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>358.000000</td>\n",
       "      <td>358.000000</td>\n",
       "      <td>358.000000</td>\n",
       "      <td>0.004253</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>5.656854</td>\n",
       "      <td>-4.842091</td>\n",
       "      <td>6.814574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-14.343715</td>\n",
       "      <td>10.827622</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000319</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-1.343715</td>\n",
       "      <td>1.900300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.991896</td>\n",
       "      <td>12.739384</td>\n",
       "      <td>715.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>0.001178</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.008104</td>\n",
       "      <td>0.011461</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.678133</td>\n",
       "      <td>4.661854</td>\n",
       "      <td>495.500000</td>\n",
       "      <td>292.000000</td>\n",
       "      <td>699.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>495.500000</td>\n",
       "      <td>292.000000</td>\n",
       "      <td>699.000000</td>\n",
       "      <td>0.025127</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.678133</td>\n",
       "      <td>2.409214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.461111</td>\n",
       "      <td>3.830581</td>\n",
       "      <td>581.000000</td>\n",
       "      <td>465.000000</td>\n",
       "      <td>697.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>581.000000</td>\n",
       "      <td>465.000000</td>\n",
       "      <td>697.000000</td>\n",
       "      <td>0.010561</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-0.538889</td>\n",
       "      <td>0.412060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>5.414251</td>\n",
       "      <td>0.030776</td>\n",
       "      <td>488.000000</td>\n",
       "      <td>278.000000</td>\n",
       "      <td>698.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>488.000000</td>\n",
       "      <td>278.000000</td>\n",
       "      <td>698.000000</td>\n",
       "      <td>0.055176</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.585749</td>\n",
       "      <td>0.030776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-0.464373</td>\n",
       "      <td>4.920910</td>\n",
       "      <td>424.500000</td>\n",
       "      <td>131.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>131.000000</td>\n",
       "      <td>131.000000</td>\n",
       "      <td>131.000000</td>\n",
       "      <td>0.063996</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.464373</td>\n",
       "      <td>2.150158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.004337</td>\n",
       "      <td>0.279581</td>\n",
       "      <td>688.500000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>701.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>688.500000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>701.000000</td>\n",
       "      <td>0.001091</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.995663</td>\n",
       "      <td>0.279581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.134556</td>\n",
       "      <td>19.608700</td>\n",
       "      <td>667.000000</td>\n",
       "      <td>616.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>616.000000</td>\n",
       "      <td>616.000000</td>\n",
       "      <td>616.000000</td>\n",
       "      <td>0.003496</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.134556</td>\n",
       "      <td>0.190291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-9.044244</td>\n",
       "      <td>18.322206</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000345</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.044244</td>\n",
       "      <td>1.476783</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.262243</td>\n",
       "      <td>3.804274</td>\n",
       "      <td>713.500000</td>\n",
       "      <td>709.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>709.000000</td>\n",
       "      <td>709.000000</td>\n",
       "      <td>709.000000</td>\n",
       "      <td>0.001002</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-2.262243</td>\n",
       "      <td>3.266793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.543169</td>\n",
       "      <td>2.154392</td>\n",
       "      <td>607.500000</td>\n",
       "      <td>600.000000</td>\n",
       "      <td>615.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>607.500000</td>\n",
       "      <td>600.000000</td>\n",
       "      <td>615.000000</td>\n",
       "      <td>0.000617</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-1.456831</td>\n",
       "      <td>2.088248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.860464</td>\n",
       "      <td>0.007328</td>\n",
       "      <td>705.500000</td>\n",
       "      <td>693.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>693.000000</td>\n",
       "      <td>693.000000</td>\n",
       "      <td>693.000000</td>\n",
       "      <td>0.000675</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.139536</td>\n",
       "      <td>0.007329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.009931</td>\n",
       "      <td>15.542305</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000374</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>-0.009931</td>\n",
       "      <td>0.014044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.556309</td>\n",
       "      <td>0.537991</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000563</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-4.556309</td>\n",
       "      <td>6.533077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.187060</td>\n",
       "      <td>5.319309</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>682.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>682.000000</td>\n",
       "      <td>682.000000</td>\n",
       "      <td>682.000000</td>\n",
       "      <td>0.000650</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.187060</td>\n",
       "      <td>1.751759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.351510</td>\n",
       "      <td>19.301880</td>\n",
       "      <td>471.000000</td>\n",
       "      <td>224.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>224.000000</td>\n",
       "      <td>224.000000</td>\n",
       "      <td>224.000000</td>\n",
       "      <td>0.026154</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.351510</td>\n",
       "      <td>0.497110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.996993</td>\n",
       "      <td>2.802835</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>0.003007</td>\n",
       "      <td>0.025593</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.353777</td>\n",
       "      <td>0.114393</td>\n",
       "      <td>593.500000</td>\n",
       "      <td>564.000000</td>\n",
       "      <td>623.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>593.500000</td>\n",
       "      <td>564.000000</td>\n",
       "      <td>623.000000</td>\n",
       "      <td>0.000926</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.646223</td>\n",
       "      <td>0.114393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.264111</td>\n",
       "      <td>5.225840</td>\n",
       "      <td>697.000000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>676.000000</td>\n",
       "      <td>0.001765</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.264110</td>\n",
       "      <td>1.845228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.110722</td>\n",
       "      <td>0.014095</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000453</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.889278</td>\n",
       "      <td>0.014095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.293358</td>\n",
       "      <td>15.141479</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000339</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-3.293357</td>\n",
       "      <td>4.657511</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.560119</td>\n",
       "      <td>3.386142</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000457</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-2.560119</td>\n",
       "      <td>3.684926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.002756</td>\n",
       "      <td>0.024730</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000506</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.002756</td>\n",
       "      <td>0.024730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.940727</td>\n",
       "      <td>0.004637</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.059273</td>\n",
       "      <td>0.004637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.983770</td>\n",
       "      <td>18.407728</td>\n",
       "      <td>694.500000</td>\n",
       "      <td>671.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>671.000000</td>\n",
       "      <td>671.000000</td>\n",
       "      <td>671.000000</td>\n",
       "      <td>0.009215</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.983771</td>\n",
       "      <td>1.391262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.415610</td>\n",
       "      <td>14.968588</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000380</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-3.415610</td>\n",
       "      <td>4.830402</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.022202</td>\n",
       "      <td>15.524952</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000504</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-3.022202</td>\n",
       "      <td>4.274038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>48</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.358756</td>\n",
       "      <td>5.108843</td>\n",
       "      <td>630.000000</td>\n",
       "      <td>542.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>542.000000</td>\n",
       "      <td>542.000000</td>\n",
       "      <td>542.000000</td>\n",
       "      <td>0.031844</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.358756</td>\n",
       "      <td>1.962225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>49</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-9.046370</td>\n",
       "      <td>18.319201</td>\n",
       "      <td>601.500000</td>\n",
       "      <td>485.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>485.000000</td>\n",
       "      <td>485.000000</td>\n",
       "      <td>485.000000</td>\n",
       "      <td>0.057393</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.046369</td>\n",
       "      <td>1.479789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.046962</td>\n",
       "      <td>5.562760</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000530</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.046962</td>\n",
       "      <td>1.508308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.492943</td>\n",
       "      <td>3.444745</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000590</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-2.492943</td>\n",
       "      <td>3.626323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>52</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>1.879464</td>\n",
       "      <td>2.450062</td>\n",
       "      <td>558.000000</td>\n",
       "      <td>398.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>398.000000</td>\n",
       "      <td>398.000000</td>\n",
       "      <td>398.000000</td>\n",
       "      <td>0.070488</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.120536</td>\n",
       "      <td>2.450062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>53</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.528001</td>\n",
       "      <td>4.845181</td>\n",
       "      <td>706.500000</td>\n",
       "      <td>695.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>695.000000</td>\n",
       "      <td>695.000000</td>\n",
       "      <td>695.000000</td>\n",
       "      <td>0.016939</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.528001</td>\n",
       "      <td>2.225886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.822007</td>\n",
       "      <td>0.646087</td>\n",
       "      <td>657.000000</td>\n",
       "      <td>645.000000</td>\n",
       "      <td>669.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>657.000000</td>\n",
       "      <td>645.000000</td>\n",
       "      <td>669.000000</td>\n",
       "      <td>0.019826</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.177993</td>\n",
       "      <td>0.646087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>55</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>3.995519</td>\n",
       "      <td>0.455112</td>\n",
       "      <td>430.500000</td>\n",
       "      <td>350.000000</td>\n",
       "      <td>511.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>430.500000</td>\n",
       "      <td>350.000000</td>\n",
       "      <td>511.000000</td>\n",
       "      <td>0.123046</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.004481</td>\n",
       "      <td>0.455112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>56</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.305325</td>\n",
       "      <td>19.367195</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000605</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.305325</td>\n",
       "      <td>0.431795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>57</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-9.867764</td>\n",
       "      <td>17.157574</td>\n",
       "      <td>561.500000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>405.000000</td>\n",
       "      <td>0.093711</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.867763</td>\n",
       "      <td>2.641416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>58</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-12.116339</td>\n",
       "      <td>13.977608</td>\n",
       "      <td>515.500000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>0.133250</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-4.116339</td>\n",
       "      <td>5.821383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>59</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-22.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000690</td>\n",
       "      <td>-2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-20.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-1.642574</td>\n",
       "      <td>1.853564</td>\n",
       "      <td>669.000000</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>0.066802</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-7.642574</td>\n",
       "      <td>1.853564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>61</td>\n",
       "      <td>0.000200</td>\n",
       "      <td>2.604650</td>\n",
       "      <td>1.165782</td>\n",
       "      <td>475.500000</td>\n",
       "      <td>362.000000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>475.500000</td>\n",
       "      <td>362.000000</td>\n",
       "      <td>589.000000</td>\n",
       "      <td>0.161822</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.395350</td>\n",
       "      <td>1.165782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>62</td>\n",
       "      <td>0.000200</td>\n",
       "      <td>3.873749</td>\n",
       "      <td>0.538616</td>\n",
       "      <td>587.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>0.162496</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.126251</td>\n",
       "      <td>0.538616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>63</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-22.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>649.000000</td>\n",
       "      <td>580.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>580.000000</td>\n",
       "      <td>580.000000</td>\n",
       "      <td>580.000000</td>\n",
       "      <td>0.118425</td>\n",
       "      <td>-2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-20.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>64</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.006168</td>\n",
       "      <td>15.547626</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.001184</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>-0.006168</td>\n",
       "      <td>0.008723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>65</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-3.888456</td>\n",
       "      <td>0.006561</td>\n",
       "      <td>609.500000</td>\n",
       "      <td>501.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>501.000000</td>\n",
       "      <td>501.000000</td>\n",
       "      <td>501.000000</td>\n",
       "      <td>0.104230</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.111544</td>\n",
       "      <td>0.006561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>66</td>\n",
       "      <td>0.000200</td>\n",
       "      <td>4.289947</td>\n",
       "      <td>0.817333</td>\n",
       "      <td>488.000000</td>\n",
       "      <td>348.000000</td>\n",
       "      <td>628.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>488.000000</td>\n",
       "      <td>348.000000</td>\n",
       "      <td>628.000000</td>\n",
       "      <td>0.164579</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.710053</td>\n",
       "      <td>0.817333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>67</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>0.639281</td>\n",
       "      <td>6.531106</td>\n",
       "      <td>615.500000</td>\n",
       "      <td>531.000000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>615.500000</td>\n",
       "      <td>531.000000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>0.114511</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-0.360719</td>\n",
       "      <td>0.539962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>68</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-9.592816</td>\n",
       "      <td>17.546408</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>522.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>522.000000</td>\n",
       "      <td>522.000000</td>\n",
       "      <td>522.000000</td>\n",
       "      <td>0.123123</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.592816</td>\n",
       "      <td>2.252582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>69</td>\n",
       "      <td>0.000200</td>\n",
       "      <td>-10.170852</td>\n",
       "      <td>16.728943</td>\n",
       "      <td>550.500000</td>\n",
       "      <td>383.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>383.000000</td>\n",
       "      <td>383.000000</td>\n",
       "      <td>383.000000</td>\n",
       "      <td>0.199826</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-2.170852</td>\n",
       "      <td>3.070048</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>70</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>2.483256</td>\n",
       "      <td>3.543573</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>545.000000</td>\n",
       "      <td>585.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>545.000000</td>\n",
       "      <td>585.000000</td>\n",
       "      <td>0.051015</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-0.516744</td>\n",
       "      <td>0.699067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>71</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-9.457619</td>\n",
       "      <td>17.737606</td>\n",
       "      <td>587.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>456.000000</td>\n",
       "      <td>0.123756</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.457618</td>\n",
       "      <td>2.061384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>72</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>3.660128</td>\n",
       "      <td>0.063749</td>\n",
       "      <td>382.000000</td>\n",
       "      <td>316.000000</td>\n",
       "      <td>448.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>382.000000</td>\n",
       "      <td>316.000000</td>\n",
       "      <td>448.000000</td>\n",
       "      <td>0.119254</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.339872</td>\n",
       "      <td>0.063749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>73</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.333180</td>\n",
       "      <td>0.206600</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.001088</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.666820</td>\n",
       "      <td>0.206600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>74</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-0.504115</td>\n",
       "      <td>0.558228</td>\n",
       "      <td>511.000000</td>\n",
       "      <td>304.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>304.000000</td>\n",
       "      <td>304.000000</td>\n",
       "      <td>304.000000</td>\n",
       "      <td>0.132574</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.504115</td>\n",
       "      <td>0.558228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>0.416099</td>\n",
       "      <td>6.221312</td>\n",
       "      <td>672.500000</td>\n",
       "      <td>627.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>627.000000</td>\n",
       "      <td>627.000000</td>\n",
       "      <td>627.000000</td>\n",
       "      <td>0.066971</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-0.583901</td>\n",
       "      <td>0.849756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>76</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.129397</td>\n",
       "      <td>5.408553</td>\n",
       "      <td>698.500000</td>\n",
       "      <td>679.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>679.000000</td>\n",
       "      <td>679.000000</td>\n",
       "      <td>679.000000</td>\n",
       "      <td>0.027459</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.129397</td>\n",
       "      <td>1.662514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>77</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>3.334516</td>\n",
       "      <td>0.131370</td>\n",
       "      <td>613.000000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>0.092074</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.665484</td>\n",
       "      <td>0.131370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>78</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.348961</td>\n",
       "      <td>0.116257</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>545.000000</td>\n",
       "      <td>587.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>545.000000</td>\n",
       "      <td>587.000000</td>\n",
       "      <td>0.040154</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.651039</td>\n",
       "      <td>0.116257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>79</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-9.434928</td>\n",
       "      <td>17.769695</td>\n",
       "      <td>650.000000</td>\n",
       "      <td>582.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>582.000000</td>\n",
       "      <td>582.000000</td>\n",
       "      <td>582.000000</td>\n",
       "      <td>0.080286</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-1.434928</td>\n",
       "      <td>2.029295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>80</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.286523</td>\n",
       "      <td>5.212905</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000974</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-1.286523</td>\n",
       "      <td>1.858163</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>81</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.967982</td>\n",
       "      <td>0.000967</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000816</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.032018</td>\n",
       "      <td>0.000967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>82</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>2.459142</td>\n",
       "      <td>3.499118</td>\n",
       "      <td>616.500000</td>\n",
       "      <td>515.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>515.000000</td>\n",
       "      <td>515.000000</td>\n",
       "      <td>515.000000</td>\n",
       "      <td>0.116355</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-0.540858</td>\n",
       "      <td>0.743523</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>83</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-7.965404</td>\n",
       "      <td>7.320003</td>\n",
       "      <td>565.500000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>623.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>565.500000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>623.000000</td>\n",
       "      <td>0.087483</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>-4.465404</td>\n",
       "      <td>6.115026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>84</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.919139</td>\n",
       "      <td>6.940118</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000957</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-0.080861</td>\n",
       "      <td>0.130949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>85</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-11.022779</td>\n",
       "      <td>15.524135</td>\n",
       "      <td>684.000000</td>\n",
       "      <td>650.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>650.000000</td>\n",
       "      <td>650.000000</td>\n",
       "      <td>650.000000</td>\n",
       "      <td>0.057623</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>-0.022780</td>\n",
       "      <td>0.032216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>86</td>\n",
       "      <td>0.000200</td>\n",
       "      <td>-0.698874</td>\n",
       "      <td>0.449502</td>\n",
       "      <td>527.500000</td>\n",
       "      <td>337.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>337.000000</td>\n",
       "      <td>337.000000</td>\n",
       "      <td>337.000000</td>\n",
       "      <td>0.150962</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.698874</td>\n",
       "      <td>0.449502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>87</td>\n",
       "      <td>0.000300</td>\n",
       "      <td>-3.024392</td>\n",
       "      <td>1.278615</td>\n",
       "      <td>491.500000</td>\n",
       "      <td>265.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>265.000000</td>\n",
       "      <td>265.000000</td>\n",
       "      <td>265.000000</td>\n",
       "      <td>0.259723</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-4.024392</td>\n",
       "      <td>5.792453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>88</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.466442</td>\n",
       "      <td>0.694252</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000671</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-7.466442</td>\n",
       "      <td>0.694252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>89</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.994384</td>\n",
       "      <td>12.735865</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000676</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.005616</td>\n",
       "      <td>0.007943</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>90</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.140543</td>\n",
       "      <td>0.145041</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000835</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.859457</td>\n",
       "      <td>0.145041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>91</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.112373</td>\n",
       "      <td>19.640070</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000770</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.112374</td>\n",
       "      <td>0.158920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>92</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.857078</td>\n",
       "      <td>0.052591</td>\n",
       "      <td>308.500000</td>\n",
       "      <td>307.000000</td>\n",
       "      <td>310.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>308.500000</td>\n",
       "      <td>307.000000</td>\n",
       "      <td>310.000000</td>\n",
       "      <td>0.003551</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.142922</td>\n",
       "      <td>0.052591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>93</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.961964</td>\n",
       "      <td>0.000780</td>\n",
       "      <td>709.000000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>700.000000</td>\n",
       "      <td>0.020648</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.038036</td>\n",
       "      <td>0.000780</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>94</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.377201</td>\n",
       "      <td>19.265547</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000827</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.377201</td>\n",
       "      <td>0.533443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>95</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.821126</td>\n",
       "      <td>6.782038</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000912</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>-0.178874</td>\n",
       "      <td>0.289030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>96</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.064993</td>\n",
       "      <td>19.707075</td>\n",
       "      <td>669.500000</td>\n",
       "      <td>655.000000</td>\n",
       "      <td>684.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>669.500000</td>\n",
       "      <td>655.000000</td>\n",
       "      <td>684.000000</td>\n",
       "      <td>0.032666</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>-0.064992</td>\n",
       "      <td>0.091913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>97</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.468154</td>\n",
       "      <td>2.118516</td>\n",
       "      <td>681.500000</td>\n",
       "      <td>662.000000</td>\n",
       "      <td>701.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>681.500000</td>\n",
       "      <td>662.000000</td>\n",
       "      <td>701.000000</td>\n",
       "      <td>0.045185</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>-1.531846</td>\n",
       "      <td>2.124125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>98</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>-1.984212</td>\n",
       "      <td>2.822144</td>\n",
       "      <td>642.000000</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>566.000000</td>\n",
       "      <td>0.122972</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-4.000000</td>\n",
       "      <td>2.828427</td>\n",
       "      <td>0.015788</td>\n",
       "      <td>0.006283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>99</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>1.111009</td>\n",
       "      <td>6.366108</td>\n",
       "      <td>389.500000</td>\n",
       "      <td>344.000000</td>\n",
       "      <td>435.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>389.500000</td>\n",
       "      <td>344.000000</td>\n",
       "      <td>435.000000</td>\n",
       "      <td>0.120002</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>7.071068</td>\n",
       "      <td>0.111009</td>\n",
       "      <td>0.704960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-12.968435</td>\n",
       "      <td>12.772561</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>718.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000805</td>\n",
       "      <td>-0.500000</td>\n",
       "      <td>2.121320</td>\n",
       "      <td>-9.500000</td>\n",
       "      <td>14.849242</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>4.242640</td>\n",
       "      <td>0.031565</td>\n",
       "      <td>0.044639</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def matmul(A, B):\n",
      "    m = len(A)\n",
      "    k = len(A[0])\n",
      "    n = len(B[0])\n",
      "    # adjust check\n",
      "    if len(B) != k: raise ValueError\n",
      "    # initialize result\n",
      "    result = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for j in range(n):\n",
      "            sum_val = 0\n",
      "            for p in range(k):\n",
      "                sum_val += A[i][p]*B[p][j]\n",
      "            result[i][j] = sum_val\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "Unsloth: Will smartly offload gradients to save VRAM!\n",
      "def matmul(A, B):\n",
      "    # A: m x n, B: n x p\n",
      "    m, n = len(A), len(A[0]) # error if A empty\n",
      "    # verify shape of B\n",
      "    if len(B) != n: raise ValueError(\"Incompatible dimensions\")\n",
      "    p = len(B[0])\n",
      "    # compute result matrix\n",
      "    result = [[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            aik = A[i][k]\n",
      "            if aik:\n",
      "                for j in range(p):\n",
      "                    result[i][j] += aik*B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B, where A and B are lists of lists.\n",
      "    The function performs a standard matrix multiplication using plain\n",
      "    Python loops and integer/float arithmetic without any external libraries.\n",
      "\n",
      "    Parameters:\n",
      "        A (list[list[Union[int, float]]]): Left\u2011hand operand.\n",
      "        B (list[list[Union[int, float]]]): Right\u2011hand operand.\n",
      "\n",
      "    Returns:\n",
      "        list[list[Union[int, float]]]: Product matrix C = A * B.\n",
      "\n",
      "    Raises:\n",
      "        ValueError: If matrix dimensions are incompatible.\n",
      "    \"\"\"\n",
      "    # Validate inputs\n",
      "    if not A or not B:\n",
      "        raise ValueError(\"Input matrices must not be empty.\")\n",
      "    if any(len(row) == 0 for row in A) or any(len(row) == 0 for row in B):\n",
      "        raise ValueError(\"Matrix rows must be non\u2011empty.\")\n",
      "    if len(A[0]) != len(B):\n",
      "        raise ValueError(\n",
      "            f\"Incompatible dimensions: A is {len(A)}x{len(A[0])} \"\n",
      "            f\"but B is {len(B)}x{len(B[0])}.\"\n",
      "        )\n",
      "\n",
      "    m = len(A)           # Rows in A\n",
      "    n = len(B[0])        # Columns in B\n",
      "    p = len(B)           # Columns in A = rows in B\n",
      "\n",
      "    # Allocate result matrix\n",
      "    C = [[0.0 for _ in range(n)] for _ in range(m)]\n",
      "\n",
      "    # Compute matrix product\n",
      "    for i in range(m):\n",
      "        for j in range(n):\n",
      "            sum_val = 0.0\n",
      "            for k in range(p):\n",
      "                sum_val += A[i][k] * B[k][j]\n",
      "            C[i][j] = sum_val\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    if not A or not B:\n",
      "        return []\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    result = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for k in range(p):\n",
      "            aik = A[i][k]\n",
      "            if aik:\n",
      "                for j in range(m):\n",
      "                    result[i][j] += aik*B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # A: m x n, B: n x p\n",
      "    m = len(A)\n",
      "    n = len(A[0])  # also len(B)\n",
      "    p = len(B[0])\n",
      "    # initialize result matrix with zeros\n",
      "    result = [[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        rowA = A[i]\n",
      "        res_row = result[i]\n",
      "        for k in range(n):\n",
      "            aik = rowA[k]\n",
      "            if aik:\n",
      "                # then we add aik * B[k][j] to each column j\n",
      "                rowBk = B[k]\n",
      "                for j in range(p):\n",
      "                    res_row[j] += aik * rowBk[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    m = len(A)\n",
      "    k = len(A[0])\n",
      "    n = len(B[0])\n",
      "    C = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        a_row = A[i]\n",
      "        Ci = C[i]\n",
      "        for j in range(n):\n",
      "            s = 0\n",
      "            for t in range(k):\n",
      "                s += a_row[t] * B[t][j]\n",
      "            Ci[j] = s\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    n = len(B[0])  # columns in B\n",
      "    result = []\n",
      "    for i in range(len(A)):\n",
      "        row = []\n",
      "        for j in range(n):\n",
      "            val = 0\n",
      "            for k in range(len(A[0])):\n",
      "                val += A[i][k] * B[k][j]\n",
      "            row.append(val)\n",
      "        result.append(row)\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # Implementation (Strassen) ...\n",
      "def matmul(A, B):\n",
      "    assert len(A[0]) == len(B)\n",
      "    m = len(A)\n",
      "    n = len(A[0])\n",
      "    p = len(B[0])\n",
      "    result = [[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        for k in range(n):\n",
      "            aik = Ai[k]\n",
      "            if aik:\n",
      "                Bk = B[k]\n",
      "                for j in range(p):\n",
      "                    result[i][j] += aik * Bk[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    # selects optimal tiling size\n",
      "    ...\n",
      "    # uses bit manipulation to accelerate computations\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    assert p == len(A[0])\n",
      "    # build transposed B for faster row dot\n",
      "    Bt = [list(col) for col in zip(*B)]\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B, where A and B are lists of lists.\n",
      "    A should be of shape (m, n) and B of shape (n, p); the result is an\n",
      "    (m, p) matrix.\n",
      "\n",
      "    Example\n",
      "    -------\n",
      "        A = [[1, 2], [3, 4]]\n",
      "        B = [[5, 6], [7, 8]]\n",
      "        >>> matmul(A, B)\n",
      "        [[19, 22], [43, 50]]\n",
      "    \"\"\"\n",
      "    # Ensure input is rectangular\n",
      "    if not A or not B or not A[0] or not B[0]:\n",
      "        return []\n",
      "\n",
      "    n_rows_A = len(A)\n",
      "    n_cols_A = len(A[0])   # number of columns in A\n",
      "    n_rows_B = len(B)\n",
      "    n_cols_B = len(B[0])   # number of columns in B\n",
      "\n",
      "    if n_cols_A != n_rows_B:\n",
      "        raise ValueError(\"Number of columns of A must equal number of rows of B\")\n",
      "\n",
      "    # Pre\u2011allocate the result matrix\n",
      "    result = [[0] * n_cols_B for _ in range(n_rows_A)]\n",
      "\n",
      "    # For better cache performance we iterate over columns of B only once\n",
      "    for i in range(n_rows_A):\n",
      "        ai = A[i]  # local reference\n",
      "        for k in range(n_cols_A):\n",
      "            aik = ai[k]\n",
      "            if aik == 0:\n",
      "                continue   # skip zero entries for a tiny speed bump\n",
      "            bk_row = B[k]\n",
      "            for j in range(n_cols_B):\n",
      "                result[i][j] += aik * bk_row[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # get dims\n",
      "    ra = len(A); ca= len(A[0]) if A else 0\n",
      "    rb = len(B); cb= len(B[0]) if B else 0\n",
      "    # check dims\n",
      "    if ca != rb: raise ValueError\n",
      "    # transpose B\n",
      "    B_T = [[B[i][j] for i in range(rb)] for j in range(cb)]\n",
      "    # compute result\n",
      "    return [[sum(a*b for a,b in zip(A[i], B_T[j])) for j in range(cb)] for i in range(ra)]\n",
      "def matmul(A, B):\n",
      "    m = len(A)\n",
      "    n = len(B[0])\n",
      "    p = len(B)\n",
      "    # B transpose to improve locality\n",
      "    B_T = list(zip(*B))\n",
      "    out = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        for j in range(n):\n",
      "            sum = 0\n",
      "            Bj = B_T[j]\n",
      "            for k in range(p):\n",
      "                sum += Ai[k]*Bj[k]\n",
      "            out[i][j] = sum\n",
      "    return out\n",
      "def matmul(A, B): return ...\n",
      "def matmul(A, B):\n",
      "    m = len(A)\n",
      "    n = len(B[0])\n",
      "    p = len(A[0])\n",
      "    assert p == len(B)\n",
      "    result = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(p):\n",
      "            aik = A[i][k]\n",
      "            for j in range(n):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # A: list of lists (m x n), B: list of lists (n x p)\n",
      "    # returns list of lists (m x p)\n",
      "    m, n = len(A), len(A[0])\n",
      "    p = len(B[0])\n",
      "    # verify dimensions\n",
      "    # compute product\n",
      "    result = [[0] * p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            aik = A[i][k]\n",
      "            for j in range(p):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    # validate\n",
      "    m = len(A)\n",
      "    n = len(B[0])  # columns of result\n",
      "    # B must be of dimension (len(A[0]) x len(B[0]))\n",
      "    BT = list(zip(*B))  # Transposed B\n",
      "    result = [[sum(x*y for x, y in zip(row, col)) for col in BT] for row in A]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    C = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        ai = A[i]\n",
      "        ci = C[i]\n",
      "        for k in range(p):\n",
      "            aik = ai[k]\n",
      "            if aik:\n",
      "                bk = B[k]\n",
      "                for j in range(m):\n",
      "                    ci[j] += aik * bk[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices represented as lists of lists using only native Python code.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list of list of numbers\n",
      "        The left matrix (m x n).\n",
      "    B : list of list of numbers\n",
      "        The right matrix (n x p).\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    list of list of numbers\n",
      "        The product matrix (m x p).\n",
      "    \"\"\"\n",
      "    # Sanity check: A must be compatible with B.\n",
      "    if not A or not B:\n",
      "        return []\n",
      "\n",
      "    m, n = len(A), len(A[0])\n",
      "    n2, p = len(B), len(B[0])\n",
      "    if n != n2:\n",
      "        raise ValueError(\"Inner dimensions of A and B must match\")\n",
      "\n",
      "    # Pre\u2011allocate the result matrix with zeros.\n",
      "    C = [[0] * p for _ in range(m)]\n",
      "\n",
      "    # Standard triple\u2011loop algorithm\n",
      "    for i in range(m):\n",
      "        ai = A[i]\n",
      "        ci = C[i]\n",
      "        for k in range(n):\n",
      "            aik = ai[k]\n",
      "            if aik == 0:\n",
      "                continue\n",
      "            bk = B[k]         # Row of B affected\n",
      "            for j in range(p):\n",
      "                ci[j] += aik * bk[j]\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    # Check dimensions\n",
      "    if not A: return []\n",
      "    m, n = len(A), len(A[0])  # number of rows in A and columns in A\n",
      "    if not B or len(B) != n: raise ValueError(\"Size mismatch\")\n",
      "    p = len(B[0])  # number of columns in B\n",
      "    # Precompute columns of B\n",
      "    B_cols = [[B[row][col] for row in range(n)] for col in range(p)]\n",
      "    result = [[sum(a_elem * b_elem for a_elem, b_elem in zip(row, col)) for col in B_cols] for row in A]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0]) # number of columns of B\n",
      "    common = len(B)\n",
      "    # initialize result matrix\n",
      "    res = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for k in range(common):\n",
      "            aik = A[i][k]\n",
      "            for j in range(m):\n",
      "                res[i][j] += aik * B[k][j]\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    # A is m x n, B is n x p,\n",
      "    # output shape m x p\n",
      "    m = len(A)\n",
      "    n = len(A[0]) if A else 0\n",
      "    # check B dims\n",
      "    if n == 0:\n",
      "        return []\n",
      "    p = len(B[0])\n",
      "\n",
      "    # preallocate result\n",
      "    C = [[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        Ci = C[i]\n",
      "        for k in range(n):\n",
      "            aik = Ai[k]\n",
      "            if aik:\n",
      "                Bk = B[k]\n",
      "                # if aik != 0 multiply\n",
      "                for j in range(p):\n",
      "                    Ci[j] += aik * Bk[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list[list[int | float]]\n",
      "        The left matrix (m \u00d7 k) to be multiplied.\n",
      "    B : list[list[int | float]]\n",
      "        The right matrix (k \u00d7 n) to be multiplied.\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    C : list[list[int | float]]\n",
      "        Result of the product A @ B  (m \u00d7 n matrix).\n",
      "    \"\"\"\n",
      "    # Basic checks\n",
      "    if not A or not B or not B[0]:\n",
      "        return []\n",
      "\n",
      "    m, k1 = len(A), len(A[0])\n",
      "    k2, n = len(B), len(B[0])\n",
      "    if k1 != k2:\n",
      "        raise ValueError(\"Inner dimensions of matrices must agree\")\n",
      "\n",
      "    # Initialize the result matrix with zeros\n",
      "    C = [[0.0] * n for _ in range(m)]\n",
      "\n",
      "    # A naive but well\u2011structured implementation that is reasonably fast\n",
      "    for i in range(m):\n",
      "        ai = A[i]\n",
      "        ci = C[i]\n",
      "        for j in range(n):\n",
      "            s = 0.0\n",
      "            for p in range(k1):\n",
      "                s += ai[p] * B[p][j]\n",
      "            ci[j] = s\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B using only plain Python code (no external libraries).\n",
      "    `A` and `B` are expected to be lists of lists, where each inner list represents a row.\n",
      "    This implementation uses a simple, straightforward algorithm with small optimisations\n",
      "    such as caching dimensions and avoiding repeated attribute lookups inside loops.\n",
      "\n",
      "    Note: This function expects that the number of columns in `A` matches the number of rows in `B`.\n",
      "    \"\"\"\n",
      "    # Validate dimensions\n",
      "    n_rows_A, n_cols_A = len(A), len(A[0])\n",
      "    n_rows_B, n_cols_B = len(B), len(B[0])\n",
      "    if n_cols_A != n_rows_B:\n",
      "        raise ValueError(\"Incompatible dimensions for matrix multiplication\")\n",
      "\n",
      "    # Pre\u2011allocate result matrix\n",
      "    result = [[0] * n_cols_B for _ in range(n_rows_A)]\n",
      "\n",
      "    # Transpose B to improve cache locality\n",
      "    B_transposed = [[B[row][col] for row in range(n_rows_B)] for col in range(n_cols_B)]\n",
      "\n",
      "    # Perform multiplication (standard algorithm)\n",
      "    for i in range(n_rows_A):\n",
      "        row_A = A[i]\n",
      "        for j in range(n_cols_B):\n",
      "            sum_val = 0\n",
      "            row_B = B_transposed[j]\n",
      "            for k in range(n_cols_A):\n",
      "                sum_val += row_A[k] * row_B[k]\n",
      "            result[i][j] = sum_val\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # Makes sure the matrices can be multiplied\n",
      "    if len(A[0]) != len(B):\n",
      "        raise ValueError(\"Number of columns in A must equal number of rows in B\")\n",
      "    \n",
      "    # Initialise result matrix with zeros\n",
      "    rows_A, cols_B, cols_A = len(A), len(B[0]), len(A[0])\n",
      "    C = [[0] * cols_B for _ in range(rows_A)]\n",
      "    \n",
      "    # Standard O(n\u00b3) matrix multiplication\n",
      "    for i in range(rows_A):\n",
      "        for k in range(cols_A):\n",
      "            aik = A[i][k]\n",
      "            # Skip if a[i][k] is zero to save some work\n",
      "            if aik == 0:\n",
      "                continue\n",
      "            for j in range(cols_B):\n",
      "                C[i][j] += aik * B[k][j]\n",
      "    \n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    try:\n",
      "        _lenA = len(A)\n",
      "        _lenB = len(B)\n",
      "        if _lenA == 0 or _lenB == 0:\n",
      "            return []\n",
      "        m = len(A[0])\n",
      "        if any(len(row)!=m for row in A):\n",
      "            raise ValueError\n",
      "        n = len(B[0])\n",
      "        if any(len(row)!=n for row in B):\n",
      "            raise ValueError\n",
      "        if m != len(B):\n",
      "            raise ValueError(\"Incompatible dimensions\")\n",
      "    except Exception:\n",
      "        raise\n",
      "    B_T = [tuple(col) for col in zip(*B)]  # transpose\n",
      "    res = [ [0]*n for _ in range(_lenA) ]\n",
      "    for i in range(_lenA):\n",
      "        rowA = A[i]\n",
      "        result_row = res[i]\n",
      "        for j in range(n):\n",
      "            colB = B_T[j]\n",
      "            s = 0\n",
      "            for k in range(m):\n",
      "                s += rowA[k] * colB[k]\n",
      "            result_row[j] = s\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    res = [[0]*m for _ in range(n)]\n",
      "    # Transpose B for faster column access\n",
      "    B_T = [list(col) for col in zip(*B)]\n",
      "    for i in range(n):\n",
      "        Ai = A[i]\n",
      "        for j in range(m):\n",
      "            res[i][j] = sum(a*b for a,b in zip(Ai, B_T[j]))\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    # ensure dimensions compatible\n",
      "    n = len(A)\n",
      "    m = len(A[0])\n",
      "    p = len(B[0])\n",
      "    # a quick check for compatibility\n",
      "    if len(B) != m:\n",
      "        raise ValueError(\"Incompatible matrix dimensions.\")\n",
      "    # initialize result matrix\n",
      "    result = [[0]*p for _ in range(n)]\n",
      "    # naive multiplication\n",
      "    for i in range(n):\n",
      "        for j in range(p):\n",
      "            sum_val = 0\n",
      "            for k in range(m):\n",
      "                sum_val += A[i][k] * B[k][j]\n",
      "            result[i][j] = sum_val\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    common = len(B)\n",
      "    # compute product\n",
      "    res = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        ai = A[i]\n",
      "        row_res = res[i]\n",
      "        for k in range(common):\n",
      "            aik = ai[k]\n",
      "            bk = B[k]\n",
      "            for j in range(m):\n",
      "                row_res[j] += aik * bk[j]\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    # Check input\n",
      "    nA = len(A)\n",
      "    mA = len(A[0]) if A else 0\n",
      "    nB = len(B)\n",
      "    mB = len(B[0]) if B else 0\n",
      "    # dims\n",
      "    if mA != nB:\n",
      "        raise ValueError(\"Incompatible dimensions.\")\n",
      "    # result dims\n",
      "    result = [[0]*mB for _ in range(nA)]\n",
      "    for i in range(nA):\n",
      "        for j in range(mB):\n",
      "            s = 0\n",
      "            for k in range(mA):\n",
      "                s += A[i][k] * B[k][j]\n",
      "            result[i][j] = s\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B (given as lists of lists).\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list[list[float]]\n",
      "        The first matrix (m x n).\n",
      "    B : list[list[float]]\n",
      "        The second matrix (n x p).\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    C : list[list[float]]\n",
      "        The product matrix (m x p).\n",
      "    \n",
      "    Raises\n",
      "    -------\n",
      "    ValueError\n",
      "        If the inner dimensions don't match.\n",
      "    \"\"\"\n",
      "    # Basic sanity check on dimensions\n",
      "    if not A or not B or not A[0] or not B[0]:\n",
      "        raise ValueError(\"Matrices must have non\u2011empty dimensions.\")\n",
      "    n = len(A[0])           # number of columns in A\n",
      "    if any(len(row) != n for row in A):\n",
      "        raise ValueError(\"All rows in A must have the same length.\")\n",
      "    if len(B) != n:\n",
      "        raise ValueError(\"Number of columns in A must equal number of rows in B.\")\n",
      "    \n",
      "    m = len(A)              # number of rows in A\n",
      "    p = len(B[0])           # number of columns in B\n",
      "    \n",
      "    # Pre\u2011compute columns of B for quicker access\n",
      "    B_t = [tuple(col) for col in zip(*B)]   # transpose: each column is a tuple\n",
      "\n",
      "    # Compute each entry of the product, using Python's max\u2011speed loops\n",
      "    C = [[sum(a * b for a, b in zip(A[i], col)) for col in B_t] for i in range(m)]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B.\n",
      "    A and B are lists of lists of numbers (i.e. 2D arrays).\n",
      "    Returns the result as a new list of lists.\n",
      "    \"\"\"\n",
      "    # Ensure matrix dimensions are compatible\n",
      "    if len(A[0]) != len(B):\n",
      "        raise ValueError(\"Incompatible matrix dimensions for multiplication.\")\n",
      "    \n",
      "    # Initialize the result matrix with zeros\n",
      "    result = [[0]*len(B[0]) for _ in range(len(A))]\n",
      "    \n",
      "    # Perform multiplication\n",
      "    for i in range(len(A)):\n",
      "        for j in range(len(B[0])):\n",
      "            for k in range(len(A[0])):\n",
      "                result[i][j] += A[i][k] * B[k][j]\n",
      "    \n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # Verify that matrices have compatible dimensions\n",
      "    if not A or not B or len(A[0]) != len(B):\n",
      "        raise ValueError(\"Number of columns in A must equal number of rows in B.\")\n",
      "    \n",
      "    # Initialize the result matrix (size: rows of A \u00d7 columns of B)\n",
      "    n_rows_a = len(A)\n",
      "    n_cols_b = len(B[0])\n",
      "    result = [[0] * n_cols_b for _ in range(n_rows_a)]\n",
      "    \n",
      "    # Iterate through rows of A and columns of B, accumulating the dot products\n",
      "    for i in range(n_rows_a):\n",
      "        for j in range(n_cols_b):\n",
      "            sum_val = 0\n",
      "            for k in range(len(A[0])):\n",
      "                sum_val += A[i][k] * B[k][j]\n",
      "            result[i][j] = sum_val\n",
      "    \n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # assume dims: A: n x m, B: m x p\n",
      "    n = len(A)\n",
      "    m = len(A[0]) if A else 0\n",
      "    p = len(B[0]) if B else 0\n",
      "    # initialize result matrix\n",
      "    C = [[0]*p for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        rowA = A[i]\n",
      "        rowC = C[i]\n",
      "        for k in range(m):\n",
      "            a = rowA[k]\n",
      "            if a != 0:  # optional optimization\n",
      "                colB = B[k]\n",
      "                for j in range(p):\n",
      "                    rowC[j] += a * colB[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    Bt = [[B[k][j] for k in range(n)] for j in range(n)]\n",
      "    res = [[0] * n for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        row = A[i]\n",
      "        for j in range(n):\n",
      "            res[i][j] = sum(row[k] * Bt[j][k] for k in range(n))\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B): return ...\n",
      "def matmul(A, B):\n",
      "    # Number of rows in A, number of columns in A (and rows in B), number of columns in B\n",
      "    n, m, p = len(A), len(A[0]), len(B[0])\n",
      "\n",
      "    # Prepare the result matrix with zeros\n",
      "    C = [[0] * p for _ in range(n)]\n",
      "\n",
      "    # Perform the standard O(n*m*p) multiplication\n",
      "    for i in range(n):\n",
      "        for k in range(m):\n",
      "            aik = A[i][k]          # element in A at row i, column k\n",
      "            for j in range(p):\n",
      "                C[i][j] += aik * B[k][j]\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B where A is an m\u00d7n matrix and B is an n\u00d7p matrix.\n",
      "    Returns the resulting m\u00d7p matrix.\n",
      "    \"\"\"\n",
      "    # Number of rows in A\n",
      "    rows_a = len(A)\n",
      "    # Number of columns in A (required to multiply with B)\n",
      "    cols_a = len(A[0]) if A else 0\n",
      "    # Number of columns in B\n",
      "    cols_b = len(B[0]) if B else 0\n",
      "    \n",
      "    # Quick check a few edge cases\n",
      "    if rows_a == 0 or cols_a == 0 or cols_b == 0:\n",
      "        return []\n",
      "\n",
      "    # Ensure that dimension compatibility holds\n",
      "    if len(B) != cols_a:\n",
      "        raise ValueError(\"Number of columns in A must equal number of rows in B\")\n",
      "\n",
      "    # Prepare the result matrix             \n",
      "    result = [[0.0 for _ in range(cols_b)] for _ in range(rows_a)]\n",
      "\n",
      "    # Matrix multiplication\n",
      "    for i in range(rows_a):\n",
      "        for k in range(cols_a):\n",
      "            aik = A[i][k]\n",
      "            for j in range(cols_b):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B): return ...\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    # assert len(A[0]) == p\n",
      "    C = [[0]*m for _ in range(n)]\n",
      "    # iterate\n",
      "    for i in range(n):\n",
      "        Ci = C[i]\n",
      "        Ai = A[i]\n",
      "        for k in range(p):\n",
      "            a = Ai[k]\n",
      "            if a != 0:\n",
      "                Bk = B[k]\n",
      "                for j in range(m):\n",
      "                    Ci[j] += a * Bk[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    # A: list of list, columns: n x m\n",
      "    # B: list of list, dimensions: m x p\n",
      "    # returns C: n x p\n",
      "    n = len(A)\n",
      "    m = len(A[0])\n",
      "    p = len(B[0])\n",
      "    # initialize result matrix\n",
      "    C = [[0]*p for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        ai = A[i]\n",
      "        for k in range(m):\n",
      "            aik = ai[k]\n",
      "            if aik:\n",
      "                bk = B[k]\n",
      "                for j in range(p):\n",
      "                    C[i][j] += aik * bk[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B using only native Python code.\n",
      "    \n",
      "    Parameters:\n",
      "    - A: List of lists where each sublist represents a row of matrix A.\n",
      "    - B: List of lists where each sublist represents a row of matrix B.\n",
      "    \n",
      "    Returns:\n",
      "    - Resulting matrix as a list of lists.\n",
      "    \"\"\"\n",
      "    if not A or not B:\n",
      "        return []\n",
      "\n",
      "    n_rows_A = len(A)\n",
      "    n_cols_A = len(A[0]) if n_rows_A > 0 else 0\n",
      "    n_rows_B = len(B)\n",
      "    n_cols_B = len(B[0]) if n_rows_B > 0 else 0\n",
      "\n",
      "    # Ensure the matrices can be multiplied\n",
      "    if n_cols_A != n_rows_B:\n",
      "        raise ValueError(\"Number of columns in A must equal number of rows in B\")\n",
      "\n",
      "    # Initialize result matrix with zeros\n",
      "    result = [[0] * n_cols_B for _ in range(n_rows_A)]\n",
      "\n",
      "    # Perform matrix multiplication\n",
      "    for i in range(n_rows_A):\n",
      "        for k in range(n_cols_A):\n",
      "            aik = A[i][k]\n",
      "            if aik == 0:\n",
      "                continue\n",
      "            for j in range(n_cols_B):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B using vanilla Python.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list of lists of numbers, shape (m, k)\n",
      "        Left matrix.\n",
      "    B : list of lists of numbers, shape (k, n)\n",
      "        Right matrix.\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    C : list of lists of numbers, shape (m, n)\n",
      "        The product A @ B.\n",
      "    \"\"\"\n",
      "    m = len(A)              # rows of A\n",
      "    k = len(A[0])           # columns of A / rows of B\n",
      "    n = len(B[0])           # columns of B\n",
      "\n",
      "    # Initialize the residual matrix with zeros.\n",
      "    C = [[0]*n for _ in range(m)]\n",
      "\n",
      "    # Perform the multiplication using the standard triple nested loop.\n",
      "    for i in range(m):\n",
      "        Ai = A[i]           # row of A\n",
      "        Ci = C[i]           # row of C we will fill\n",
      "        for l in range(k):     # over columns of A and rows of B\n",
      "            a = Ai[l]\n",
      "            if a != 0:          # skip zero terms to reduce work\n",
      "                Bl = B[l]\n",
      "                for j in range(n):\n",
      "                    Ci[j] += a * Bl[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B (given as lists of lists) without using external libraries.\n",
      "    \"\"\"\n",
      "    # Basic dimension checks\n",
      "    if not A or not B:\n",
      "        return []\n",
      "    m, p = len(A), len(A[0])         # A is m\u00d7p\n",
      "    assert len(B) == p                # B must be p\u00d7n\n",
      "    n = len(B[0])                     # n columns in B\n",
      "\n",
      "    # Transpose B to improve cache locality\n",
      "    B_T = list(zip(*B))               # B^T is n\u00d7p, each element is a tuple\n",
      "\n",
      "    result = [[0]*n for _ in range(m)]\n",
      "\n",
      "    for i in range(m):\n",
      "        row_A = A[i]\n",
      "        row_res = result[i]\n",
      "        for k in range(p):          # iterate over inner dimension\n",
      "            aik = row_A[k]\n",
      "            if aik:\n",
      "                col_B = B_T[k]       # k-th column of B\n",
      "                for j in range(n):\n",
      "                    row_res[j] += aik * col_B[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"Multiply two matrices A and B.\n",
      "\n",
      "    The function assumes that A and B are compatible for matrix multiplication,\n",
      "    i.e. if A is MxK then B must be KxN. The result is an MxN matrix.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list[list[float]]\n",
      "        MxK matrix.\n",
      "    B : list[list[float]]\n",
      "        KxN matrix.\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    list[list[float]]\n",
      "        Product matrix of shape MxN.\n",
      "    \"\"\"\n",
      "    # Grab sizes locally for speed\n",
      "    m = len(A)           # number of rows of A\n",
      "    k = len(A[0]) if A else 0  # number of columns in A (inner dimension)\n",
      "    n = len(B[0]) if B else 0  # number of columns in B\n",
      "\n",
      "    # Prepare the result matrix.\n",
      "    # Use a list of lists pre\u2011filled with zeros.\n",
      "    result = [[0.0] * n for _ in range(m)]\n",
      "\n",
      "    # Basic algorithm \u2013 triple nested loop.\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        Ri = result[i]\n",
      "        # Pre\u2011localize B for a bit of speed.\n",
      "        for j in range(n):\n",
      "            s = 0.0\n",
      "            for p in range(k):\n",
      "                s += Ai[p] * B[p][j]\n",
      "            Ri[j] = s\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B where A is mxk and B is kxn\n",
      "    using simple Python list-of-lists and a few optimizations.\n",
      "    Arguments:\n",
      "      A: list of lists, where A[i][j] is the entry of row i and column j\n",
      "      B: list of lists, where B[i][j] is the entry of row i and column j\n",
      "    Returns:\n",
      "      C: the resulting matrix (mxn)\n",
      "    \"\"\"\n",
      "    m, k = len(A), len(A[0])  # size of A: m rows, k columns\n",
      "    k2, n = len(B), len(B[0])  # size of B: k' rows, n columns\n",
      "    if k != k2:\n",
      "        raise ValueError(\"A and B have incompatible dimensions\")\n",
      "\n",
      "    # To speed up innermost loops we transpose B.\n",
      "    B_T = [[B[row][col] for row in range(k)] for col in range(n)]\n",
      "\n",
      "    # Initialize output matrix\n",
      "    C = [[0] * n for _ in range(m)]\n",
      "\n",
      "    for i in range(m):\n",
      "        a_row = A[i]\n",
      "        for j in range(n):\n",
      "            # compute dot product of A[i] and B_T[j] as one row\n",
      "            acc = 0\n",
      "            for t in range(k):\n",
      "                acc += a_row[t] * B_T[j][t]\n",
      "            C[i][j] = acc\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"Fast matrix multiplication using only native Python code.\n",
      "\n",
      "    The function expects A (m x n) and B (n x p) to be lists of lists.\n",
      "    It uses a single loop to compute the result efficiently.\n",
      "\n",
      "    Time complexity: O(m*n*p) in the worst case, but many Python\n",
      "    implementations can handle small matrices quickly.\n",
      "    \"\"\"\n",
      "    m, n = len(A), len(A[0])\n",
      "    nB, p = len(B), len(B[0])\n",
      "    if n != nB:\n",
      "        raise ValueError(\"A's column count must equal B's row count\")\n",
      "\n",
      "    # Initialize a zero matrix for the result\n",
      "    C = [[0.0] * p for _ in range(m)]\n",
      "    \n",
      "    # Main multiplication loop\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            aik = A[i][k]\n",
      "            if aik == 0:\n",
      "                continue\n",
      "            for j in range(p):\n",
      "                C[i][j] += aik * B[k][j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    m, n = len(A), len(A[0])  # n must equal len(B)\n",
      "    p = len(B[0])\n",
      "    return [[sum(a*b for a,b in zip(row, col)) for col in zip(*B)] for row in A]\n",
      "def matmul(A, B):\n",
      "    # check dimensions\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    # Transpose B\n",
      "    B_T = list(zip(*B))\n",
      "    result = [ [0]*m for _ in range(n) ]\n",
      "    for i, row in enumerate(A):\n",
      "        for j, col in enumerate(B_T):\n",
      "            result[i][j] = sum(a*b for a,b in zip(row, col))\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(A[0])\n",
      "    p = len(B[0])\n",
      "    result = [[0] * p for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        ai = A[i]\n",
      "        for k in range(m):\n",
      "            aik = ai[k]\n",
      "            if aik:\n",
      "                bj = B[k]\n",
      "                for j in range(p):\n",
      "                    result[i][j] += aik * bj[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0]) # columns of B\n",
      "    p = len(B) # rows of B\n",
      "    # ensure A's columns equal B's rows\n",
      "    assert len(A[0]) == len(B), \"Incompatible dimensions\"\n",
      "    result = [[0] * m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for k in range(len(B)):\n",
      "            aik = A[i][k]\n",
      "            # Multiply aik with each value in row k of B\n",
      "            for j in range(m):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    # Validate shapes\n",
      "    n = len(A)\n",
      "    m = len(A[0]) # width of A\n",
      "    p = len(B[0])\n",
      "    # maybe check that all rows are same length\n",
      "    # Also, ensure len(B) == m\n",
      "    if len(B) != m:\n",
      "        raise ValueError('Incompatible matrix shapes for multiplication.')\n",
      "    # Use maybe list comprehension for each element\n",
      "    result = [[sum(A[i][k]*B[k][j] for k in range(m)) for j in range(p)] for i in range(n)]\n",
      "    return result\n",
      "def matmul(A, B): return ...\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B provided as lists of lists.\n",
      "    A and B should be compatible for matrix multiplication\n",
      "    (number of columns of A equals number of rows of B).\n",
      "\n",
      "    Args:\n",
      "        A: List of rows, where each row is an iterable of numbers and\n",
      "           all rows have the same length.\n",
      "        B: Same format.\n",
      "\n",
      "    Returns:\n",
      "        A new list of lists containing the product A * B.\n",
      "    \"\"\"\n",
      "    # Validate dimensions\n",
      "    if not A or not B:\n",
      "        raise ValueError(\"Input matrices cannot be empty\")\n",
      "    n_rows_a = len(A)\n",
      "    n_cols_a = len(A[0])\n",
      "    n_rows_b = len(B)\n",
      "    n_cols_b = len(B[0])\n",
      "\n",
      "    if n_cols_a != n_rows_b:\n",
      "        raise ValueError(\"Incompatible dimensions for matrix multiplication\")\n",
      "\n",
      "    # Transpose B once to improve cache locality\n",
      "    B_T = [[B[row][col] for row in range(n_rows_b)] for col in range(n_cols_b)]\n",
      "\n",
      "    # Allocate result matrix\n",
      "    result = [[0] * n_cols_b for _ in range(n_rows_a)]\n",
      "\n",
      "    # Perform multiplication\n",
      "    for i in range(n_rows_a):\n",
      "        row_a = A[i]\n",
      "        row_res = result[i]\n",
      "        for j in range(n_cols_b):\n",
      "            col_b = B_T[j]\n",
      "            s = 0\n",
      "            # dot product of row_a and col_b\n",
      "            for k in range(n_cols_a):\n",
      "                s += row_a[k] * col_b[k]\n",
      "            row_res[j] = s\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n, m = len(A), len(B[0])\n",
      "    # assume A rows by k, B columns by k\n",
      "    # B's columns: B_col = [list of column values]\n",
      "    B_T = list(zip(*B))\n",
      "    C = []\n",
      "    for row in A:\n",
      "        newrow = []\n",
      "        for col in B_T:\n",
      "            newrow.append(sum(a*b for a,b in zip(row, col)))\n",
      "        C.append(newrow)\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    # Check dimensions\n",
      "    m, n = len(A), len(A[0]); p, q = len(B), len(B[0])\n",
      "    if n != p:\n",
      "        raise ValueError(\"Incompatible dimensions.\")\n",
      "    result = [[0]*q for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            aik = A[i][k]\n",
      "            for j in range(q):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"Multiply two matrices A (n x m) and B (m x p) using only native Python.\"\"\"\n",
      "    n = len(A)\n",
      "    m = len(A[0]) if A else 0\n",
      "    p = len(B[0]) if B else 0\n",
      "\n",
      "    # Verify dimensions\n",
      "    if not A or not B or len(B) != m:\n",
      "        raise ValueError(\"Incompatible matrix dimensions.\")\n",
      "\n",
      "    # Pre-allocate result matrix\n",
      "    result = [[0] * p for _ in range(n)]\n",
      "\n",
      "    for i in range(n):\n",
      "        Ai = A[i]\n",
      "        res_row = result[i]\n",
      "        for k in range(m):\n",
      "            aik = Ai[k]\n",
      "            if aik:\n",
      "                Bk = B[k]\n",
      "                for j in range(p):\n",
      "                    res_row[j] += aik * Bk[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    C = [[0.0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        a_row = A[i]\n",
      "        for k in range(p):\n",
      "            aik = a_row[k]\n",
      "            if aik!=0:\n",
      "                B_col = B[k]\n",
      "                for j in range(m):\n",
      "                    C[i][j] += aik * B_col[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    # B must have dimension matching\n",
      "    # Let's compute transpose of B for cache-friendly.\n",
      "    BT = list(map(list, zip(*B)))  # transpose B\n",
      "    result = [[sum(a*b for a,b in zip(rowA, colB)) for colB in BT] \n",
      "               for rowA in A]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    if not A or not B: return []\n",
      "    m, n = len(A), len(A[0])\n",
      "    p = len(B[0])\n",
      "    # check B's rows equal to A's columns\n",
      "    if len(B) != n: raise ValueError(\"...\")\n",
      "\n",
      "    result = [[0.0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        ai = A[i]\n",
      "        for k, a in enumerate(ai):\n",
      "            if a:\n",
      "                bk = B[k]\n",
      "                for j, b in enumerate(bk):\n",
      "                    result[i][j] += a*b\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(A[0])\n",
      "    p = len(B[0])\n",
      "def matmul(A, B): return ...\n",
      "def matmul(A, B):    return ...\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    C = [[0]*n for _ in range(n)]\n",
      "    # naive loop\n",
      "    for i in range(n):\n",
      "        for k in range(n):\n",
      "            for j in range(n):\n",
      "                C[i][j] += A[i][k] * B[k][j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    m=len(A)\n",
      "    n=len(A[0])\n",
      "    p=len(B[0])\n",
      "    C=[[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            for j in range(p):\n",
      "                C[i][j]+=A[i][k]*B[k][j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    # A: m x k, B: k x n -> result m x n\n",
      "    m = len(A)\n",
      "    k = len(A[0]) if A else 0\n",
      "    n = len(B[0]) if B else 0\n",
      "    # Precompute B's column representation\n",
      "    Bt = list(zip(*B))  # columns of B as tuples\n",
      "    return [[sum(a*b for a,b in zip(row, col)) for col in Bt] for row in A]\n",
      "def matmul(A, B):\n",
      "    import numpy as np\n",
      "    A_arr = np.array(A)\n",
      "    B_arr = np.array(B)\n",
      "    return (A_arr @ B_arr).tolist()\n",
      "def matmul(A, B):\n",
      "    n=len(A); m=len(B[0]); p=len(B) # that is dimension of B's rows\n",
      "    # use zeros list for result\n",
      "    result=[[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        a_row=A[i]\n",
      "        for j in range(m):\n",
      "           s=0\n",
      "           for k in range(p):\n",
      "                s+=a_row[k]*B[k][j]\n",
      "           result[i][j]=s\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B.\n",
      "    A is (m \u00d7 n) and B is (n \u00d7 p), both represented as lists of lists.\n",
      "    Returns the product matrix of dimension (m \u00d7 p).\n",
      "    This routine is written purely in Python and uses a little bit of\n",
      "    pre\u2011processing to keep memory accesses cache\u2011friendly.\n",
      "    \"\"\"\n",
      "    # dimensions\n",
      "    m, n = len(A), len(A[0])\n",
      "    nB, p = len(B), len(B[0])\n",
      "    assert n == nB, \"Inner dimensions must agree\"\n",
      "\n",
      "    # transpose B to keep the inner loop cache\u2011friendly\n",
      "    B_T = [[B[k][j] for k in range(n)] for j in range(p)]\n",
      "\n",
      "    # prepare result matrix\n",
      "    C = [[0] * p for _ in range(m)]\n",
      "\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        Ci = C[i]\n",
      "        for k in range(n):\n",
      "            aik = Ai[k]\n",
      "            if aik == 0:\n",
      "                continue\n",
      "            Bk = B_T[k]\n",
      "            for j in range(p):\n",
      "                Ci[j] += aik * Bk[j]\n",
      "\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B using only native Python code.\n",
      "\n",
      "    Parameters:\n",
      "        A: list of lists, shape (m, n)\n",
      "        B: list of lists, shape (n, p)\n",
      "\n",
      "    Returns:\n",
      "        C: list of lists, shape (m, p)\n",
      "    \"\"\"\n",
      "    # check dimensions\n",
      "    m, n = len(A), len(A[0])\n",
      "    if len(B) != n:\n",
      "        raise ValueError(\"Inner matrix dimensions do not match\")\n",
      "    p = len(B[0])\n",
      "\n",
      "    # pre\u2011allocate the result matrix\n",
      "    C = [[0] * p for _ in range(m)]\n",
      "\n",
      "    # multiply\n",
      "    for i in range(m):\n",
      "        ai = A[i]\n",
      "        for j in range(p):\n",
      "            s = 0\n",
      "            # iterate over the shared dimension\n",
      "            for k in range(n):\n",
      "                s += ai[k] * B[k][j]\n",
      "            C[i][j] = s\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    m, n_A = len(A), len(A[0])\n",
      "    n_B, p = len(B), len(B[0])\n",
      "    assert n_A == n_B  # The matrices must have compatible dimensions\n",
      "    \n",
      "    # Preallocate the output matrix\n",
      "    result = [[0.0] * p for _ in range(m)]\n",
      "    \n",
      "    for i in range(m):\n",
      "        for j in range(p):\n",
      "            # Compute the dot product of row i and column j\n",
      "            sum_val = 0.0\n",
      "            for k in range(n_A):  # or n_B\n",
      "                sum_val += A[i][k] * B[k][j]\n",
      "            result[i][j] = sum_val\n",
      "    \n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    # Assume A and B are both n x n\n",
      "    # If n=1 return element-wise product\n",
      "    # else compute.\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiplies two matrices A and B.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list of lists\n",
      "        First matrix, with dimensions m x n.\n",
      "    B : list of lists\n",
      "        Second matrix, with dimensions n x p.\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    list of lists\n",
      "        Resulting matrix of dimensions m x p.\n",
      "    \"\"\"\n",
      "    # Number of rows in A and columns in B\n",
      "    m = len(A)\n",
      "    n = len(A[0])   # Shared dimension\n",
      "    p = len(B[0])   # Columns in B\n",
      "\n",
      "    # Prepare result matrix filled with zeros\n",
      "    C = [[0 for _ in range(p)] for _ in range(m)]\n",
      "\n",
      "    # Standard triple\u2011loop multiplication\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            aik = A[i][k]\n",
      "            for j in range(p):\n",
      "                C[i][j] += aik * B[k][j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    n_rows_A = len(A)\n",
      "    n_cols_A = len(A[0]) if A else 0\n",
      "    n_rows_B = len(B)\n",
      "    n_cols_B = len(B[0]) if B else 0\n",
      "    if n_cols_A != n_rows_B:\n",
      "        raise ValueError(\"Incompatible dimensions\")\n",
      "    C = [[0]*n_cols_B for _ in range(n_rows_A)]\n",
      "    for i in range(n_rows_A):\n",
      "        Ai = A[i]\n",
      "        Ci = C[i]\n",
      "        for k in range(n_cols_A):\n",
      "            aik = Ai[k]\n",
      "            if aik:\n",
      "                Bk = B[k]\n",
      "                for j in range(n_cols_B):\n",
      "                    Ci[j] += aik * Bk[j]\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    m, n = len(A), len(B[0]) \n",
      "    # etc...\n",
      "    # compute product\n",
      "    result = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(len(A[0])):  # iterate columns of A\n",
      "            aik = A[i][k]\n",
      "            for j in range(n):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    import math\n",
      "    n = len(A)\n",
      "    m = len(A[0])\n",
      "    # Validate B shape: m==len(B)\n",
      "    assert m == len(B), \"Incompatible mat shape\"\n",
      "    # optionally convert to square by padding with zeros for Strassen\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(A[0])  # number of columns in A\n",
      "    p = len(B[0])\n",
      "    # Pre-allocate result matrix\n",
      "    C = [[0]*p for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        row = A[i]\n",
      "        for j in range(p):\n",
      "            s = 0\n",
      "            for k in range(m):\n",
      "                s += row[k]*B[k][j]\n",
      "            C[i][j] = s\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])  # columns of B\n",
      "    result = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for k in range(len(A[i])):\n",
      "            aik = A[i][k]\n",
      "            if aik != 0:\n",
      "                for j in range(m):\n",
      "                    result[i][j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\\n     return ...\n",
      "def matmul(A, B):\n",
      "    n, m = len(A), len(A[0])\n",
      "    o, p = len(B), len(B[0])\n",
      "    assert m == o\n",
      "    # Optionally use comprehension:\n",
      "    result = [[sum(A[i][k]*B[k][j] for k in range(m)) for j in range(p)] for i in range(n)]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A: list of lists (rows of A)\n",
      "    B: list of lists (rows of B)\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    Resulting matrix as a list of lists.\n",
      "    \"\"\"\n",
      "    # Determine dimensions\n",
      "    m = len(A)           # number of rows in A\n",
      "    k = len(A[0]) if A else 0    # number of columns in A\n",
      "    n = len(B[0])     # number of columns in B\n",
      "\n",
      "    # Initialize result matrix\n",
      "    C = [[0] * n for _ in range(m)]\n",
      "\n",
      "    # Standard triple-loop matrix multiplication\n",
      "    for i in range(m):\n",
      "        for j in range(n):\n",
      "            s = 0\n",
      "            for l in range(k):\n",
      "                s += A[i][l] * B[l][j]\n",
      "            C[i][j] = s\n",
      "    return C\n",
      "def matmul(A, B):\n",
      "    # sizes: A is n\u00d7p , B is p\u00d7m\n",
      "    n, p = len(A), len(A[0])          # number of rows of A, columns of A\n",
      "    mp = len(B[0])                    # number of columns of B\n",
      "    # prepare result matrix n\u00d7m and fill it\n",
      "    R = [[0] * mp for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for j in range(mp):\n",
      "            s = 0\n",
      "            for k in range(p):\n",
      "                s += A[i][k] * B[k][j]\n",
      "            R[i][j] = s\n",
      "    return R\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "def matmul(A, B):\n",
      "    n, m = len(A), len(B[0])\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    result = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for j in range(m):\n",
      "            s = 0\n",
      "            for k in range(p):\n",
      "                s += A[i][k] * B[k][j]\n",
      "            result[i][j] = s\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    n = len(A)\n",
      "    m = len(B[0])\n",
      "    p = len(B)\n",
      "    result = [[0]*m for _ in range(n)]\n",
      "    for i in range(n):\n",
      "        for k in range(p):\n",
      "            aik = A[i][k]\n",
      "            if aik:\n",
      "                row = result[i]\n",
      "                for j in range(m):\n",
      "                    row[j] += aik * B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    if not A or not B: return []\n",
      "    m, n1 = len(A), len(A[0])\n",
      "    n2, p = len(B), len(B[0])\n",
      "    assert n1 == n2, \"Dimensions mismatch.\"\n",
      "    res = [[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n1):\n",
      "            a = A[i][k]\n",
      "            if a:\n",
      "                for j in range(p):\n",
      "                    res[i][j] += a*B[k][j]\n",
      "    return res\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B using a fast algorithm (divide\u2011and\u2011conquer).\n",
      "    The algorithm recursively multiplies submatrices. This implementation\n",
      "    does not use any external libraries (no numpy etc.) and only relies on\n",
      "    pure Python data structures.\n",
      "\n",
      "    Parameters\n",
      "    ----------\n",
      "    A : list of list of numbers\n",
      "        The left matrix.\n",
      "    B : list of list of numbers\n",
      "        The right matrix.\n",
      "\n",
      "    Returns\n",
      "    -------\n",
      "    list of list of numbers\n",
      "        The product matrix A * B.\n",
      "\n",
      "    The function assumes that A's columns equal B's rows.\n",
      "    \"\"\"\n",
      "    # Determine matrix dimensions\n",
      "    n = len(A)                # rows of A\n",
      "    m = len(B[0])             # columns of B\n",
      "    k = len(B)                # columns of A / rows of B\n",
      "\n",
      "    # A and B must be compatible\n",
      "    if not all(len(row) == k for row in A):\n",
      "        raise ValueError(\"Incompatible dimensions for matrix multiplication\")\n",
      "\n",
      "    # Base case: if matrices are small, compute directly\n",
      "    if n <= 1 or k <= 1 or m <= 1:\n",
      "        # Direct quadratic multiplication\n",
      "        result = [[0.0] * m for _ in range(n)]\n",
      "        for i in range(n):\n",
      "            for j in range(m):\n",
      "                for s in range(k):\n",
      "                    result[i][j] += A[i][s] * B[s][j]\n",
      "        return result\n",
      "\n",
      "    # Divide matrices into quadrants\n",
      "    # Helper function to split matrix into four sub\u2011matrices\n",
      "    def split(mat):\n",
      "        half_rows = len(mat) // 2\n",
      "        half_cols = len(mat[0]) // 2\n",
      "        top_left = [row[:half_cols] for row in mat[:half_rows]]\n",
      "        top_right = [row[half_cols:] for row in mat[:half_rows]]\n",
      "        bottom_left = [row[:half_cols] for row in mat[half_rows:]]\n",
      "        bottom_right = [row[half_cols:] for row in mat[half_rows:]]\n",
      "        return top_left, top_right, bottom_left, bottom_right\n",
      "\n",
      "    # Split A\n",
      "    A11, A12, A21, A22 = split(A)\n",
      "    # Split B\n",
      "    B11, B12, B21, B22 = split(B)\n",
      "\n",
      "    # Recursive multiplication for each submatrix product\n",
      "    C11 = matmul(A11, B11)\n",
      "    C12 = matmul(A12, B12)\n",
      "    C21 = matmul(A21, B21)\n",
      "    C22 = matmul(A22, B22)\n",
      "\n",
      "    # Combine the submatrices into a single result\n",
      "    result = []\n",
      "    for i in range(len(C11)):\n",
      "        result.append(C11[i] + C12[i])   # merge rows from left and right halves\n",
      "    for i in range(len(C21)):\n",
      "        result.append(C21[i] + C22[i])\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    if not A or not B or not B[0]:\n",
      "        return []\n",
      "    m, p = len(A), len(A[0])\n",
      "    p2, n = len(B), len(B[0])\n",
      "    if p != p2:\n",
      "        raise ValueError(\"A's columns must equal B's rows\")\n",
      "    # initialize result matrix\n",
      "    result = [[0]*n for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        Ai = A[i]\n",
      "        for k in range(p):\n",
      "            aik = Ai[k]\n",
      "            if aik:\n",
      "                Bk = B[k]\n",
      "                for j in range(n):\n",
      "                    result[i][j] += aik * Bk[j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # A is m x n, B is n x p\n",
      "    if not A or not B: return []\n",
      "    n = len(A[0])\n",
      "    assert all(len(row)==n for row in A)\n",
      "    assert all(len(row)==len(A[0]) for row in B)\n",
      "    m=len(A); p=len(B[0])\n",
      "    result=[[0]*p for _ in range(m)]\n",
      "    for i in range(m):\n",
      "        for k in range(n):\n",
      "            a=A[i][k]\n",
      "            if a:\n",
      "                # inner multiplication contributed to each column j\n",
      "                for j in range(p):\n",
      "                    result[i][j] += a*B[k][j]\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    ...\n",
      "def matmul(A, B):\n",
      "    # Sanity check for proper dimensions\n",
      "    n = len(A)          # Rows in A\n",
      "    m = len(A[0]) if A else 0   # Columns in A (and rows in B)\n",
      "    p = len(B[0]) if B else 0   # Columns in B\n",
      "\n",
      "    # We need A's columns to equal B's rows\n",
      "    if m != len(B):\n",
      "        raise ValueError(\"Dimension mismatch: A's columns must equal B's rows\")\n",
      "\n",
      "    # Initialize the result matrix\n",
      "    result = [[0] * p for _ in range(n)]\n",
      "\n",
      "    # Perform the multiplication\n",
      "    for i in range(n):\n",
      "        for k in range(m):\n",
      "            aik = A[i][k]       # Cache the value to avoid repeated indexing\n",
      "            for j in range(p):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "\n",
      "    return result\n",
      "def matmul(A, B):\n",
      "    # asserts\n",
      "def matmul(A, B):\n",
      "    ... # code\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=100, training_loss=3.8481087469079966e-05, metrics={'train_runtime': 46507.4754, 'train_samples_per_second': 0.004, 'train_steps_per_second': 0.002, 'total_flos': 0.0, 'train_loss': 3.8481087469079966e-05})"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tlaUdxC_VHpz"
   },
   "source": [
    "<a name=\"Inference\"></a>\n",
    "# Inference\n",
    "Now let's try the model we just trained!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "8BZZHOKiF9Ct",
    "outputId": "67906bb2-a8dc-45b7-c657-ee97d87b7e5c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\n",
      "Knowledge cutoff: 2024-06\n",
      "Current date: 2025-11-22\n",
      "\n",
      "Reasoning: low\n",
      "\n",
      "# Valid channels: analysis, commentary, final. Channel must be included for every message.\n",
      "Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Create a new fast matrix multiplication function using only native Python code.\n",
      "You are given a list of list of numbers.\n",
      "Output your new function in backticks using the format below:\n",
      "```python\n",
      "def matmul(A, B):\n",
      "    return ...\n",
      "```<|end|><|start|>assistant<|channel|>analysis<|message|>We need to output a function that multiplies two matrices, using only native Python. Maybe naive algorithm. Provide code.<|end|><|start|>assistant<|channel|>final<|message|>```python\n",
      "def matmul(A, B):\n",
      "    \"\"\"\n",
      "    Multiply two matrices A and B (as lists of lists) using the standard O(n^3) algorithm.\n",
      "    Assumes that A and B are compatible for multiplication (i.e., len(A[0]) == len(B)).\n",
      "    \"\"\"\n",
      "    # Dimensions\n",
      "    m = len(A)          # number of rows in A\n",
      "    n = len(B[0])       # number of columns in B\n",
      "    p = len(B)          # number of rows in B\n",
      "\n",
      "    # Initialize result matrix with zeros\n",
      "    result = [[0 for _ in range(n)] for _ in range(m)]\n",
      "\n",
      "    # Perform multiplication\n",
      "    for i in range(m):\n",
      "        for k in range(p):\n",
      "            aik = A[i][k]\n",
      "            for j in range(n):\n",
      "                result[i][j] += aik * B[k][j]\n",
      "\n",
      "    return result\n",
      "```<|return|>\n"
     ]
    }
   ],
   "source": [
    "text = tokenizer.apply_chat_template(\n",
    "    [{\"role\": \"user\", \"content\": prompt}],\n",
    "    tokenize = False,\n",
    "    add_generation_prompt = True,\n",
    "    reasoning_effort = \"low\",\n",
    ")\n",
    "\n",
    "from transformers import TextStreamer\n",
    "_ = model.generate(\n",
    "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
    "    temperature = 1.0,\n",
    "    max_new_tokens = 1024,\n",
    "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-NUEmHFSYNTp"
   },
   "source": [
    "<a name=\"Save\"></a>\n",
    "### Saving to float16 or MXFP4 for VLLM\n",
    "\n",
    "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `mxfp4` for MXFP4 (OpenAI's GPT-OSS native precision). We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "NjXGTkp7YNtB"
   },
   "outputs": [],
   "source": [
    "# Merge and push to hub in mxfp4 4bit format\n",
    "if False:\n",
    "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"mxfp4\")\n",
    "if False: model.push_to_hub_merged(\"repo_id/repo_name\", tokenizer, token = \"hf...\", save_method = \"mxfp4\")\n",
    "\n",
    "# Merge and push to hub in 16bit\n",
    "if False:\n",
    "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"merged_16bit\")\n",
    "if False: # Pushing to HF Hub\n",
    "    model.push_to_hub_merged(\"hf/gpt-oss-finetune\", tokenizer, save_method = \"merged_16bit\", token = \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V15Yhj1V9lwG"
   },
   "source": [
    "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
    "\n",
    "Some other links:\n",
    "1. Train your own reasoning model - Llama GRPO notebook [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb)\n",
    "2. Saving finetunes to Ollama. [Free notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)\n",
    "3. Llama 3.2 Vision finetuning - Radiography use case. [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb)\n",
    "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://docs.unsloth.ai/get-started/unsloth-notebooks)!\n",
    "\n",
    "<div class=\"align-center\">\n",
    "  <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
    "  <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
    "  <a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
    "\n",
    "  Join Discord if you need help + \u2b50\ufe0f <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> \u2b50\ufe0f\n",
    "</div>\n",
    "\n",
    "  This notebook and all Unsloth notebooks are licensed [LGPL-3.0](https://github.com/unslothai/notebooks?tab=LGPL-3.0-1-ov-file#readme).\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": [],
   "include_colab_link": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "61f544c65be943de8aed4f946f602901": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_aaeb4518db91427a9901bdb2c21eeae8",
       "IPY_MODEL_6d502f652bad4f57bb19e4f0b42ecf5e",
       "IPY_MODEL_eeccf5e7500b43279ddf30458d70633f"
      ],
      "layout": "IPY_MODEL_8440ebd6c39841da8fb353dcfd4847fd"
     }
    },
    "aaeb4518db91427a9901bdb2c21eeae8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e84d4076c4654fb38fae2803edf27894",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_3292953108134c18810e60670dfccca2",
      "value": "model.safetensors.index.json:\u2007"
     }
    },
    "6d502f652bad4f57bb19e4f0b42ecf5e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_61d0dea447c2471ba0376e034050929f",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a8ea0a0c8e8d4f29b3c84b4fc2da4f05",
      "value": 1
     }
    },
    "eeccf5e7500b43279ddf30458d70633f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca37992f3614458ca31cd16286c01264",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_50fb1f83836145ceaf582d97ca4f9164",
      "value": "\u20071.19M/?\u2007[00:00&lt;00:00,\u200757.8MB/s]"
     }
    },
    "8440ebd6c39841da8fb353dcfd4847fd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e84d4076c4654fb38fae2803edf27894": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3292953108134c18810e60670dfccca2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "61d0dea447c2471ba0376e034050929f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "a8ea0a0c8e8d4f29b3c84b4fc2da4f05": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ca37992f3614458ca31cd16286c01264": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "50fb1f83836145ceaf582d97ca4f9164": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f799530a873d4be9af603583cddd81b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fa8b093ece434ddc9e48b7fc405bab1f",
       "IPY_MODEL_169146551bae436786251cda3874ddb9",
       "IPY_MODEL_cb181b69bf024f30910f82e6be8f809c"
      ],
      "layout": "IPY_MODEL_08ecda69e4dc49c68a259aebc2fc18d3"
     }
    },
    "fa8b093ece434ddc9e48b7fc405bab1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e318dc8254564cebb6fe45029415fbde",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_ffbd26d2d0044fbcbb815ad7c52eb01b",
      "value": "model-00001-of-00004.safetensors:\u2007100%"
     }
    },
    "169146551bae436786251cda3874ddb9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2fc8f8438e2445feab841785ce7604ba",
      "max": 3998751275,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_463bb18959114ff3b35ded08d2e4bdc6",
      "value": 3998751275
     }
    },
    "cb181b69bf024f30910f82e6be8f809c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1b8ff215ef5e4467b580786577131aa3",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_b00d519503a3442191b6b784e88db3fc",
      "value": "\u20074.00G/4.00G\u2007[00:32&lt;00:00,\u200748.1MB/s]"
     }
    },
    "08ecda69e4dc49c68a259aebc2fc18d3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e318dc8254564cebb6fe45029415fbde": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ffbd26d2d0044fbcbb815ad7c52eb01b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2fc8f8438e2445feab841785ce7604ba": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "463bb18959114ff3b35ded08d2e4bdc6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1b8ff215ef5e4467b580786577131aa3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b00d519503a3442191b6b784e88db3fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a84974f9032c4e4ea3f99f530bc21fe3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_927dd36866c246d0b82c5aa91d8276cd",
       "IPY_MODEL_81dccaed1bc647609d5d423aebac4905",
       "IPY_MODEL_da7d0349c2e54258950debfd5f895c6b"
      ],
      "layout": "IPY_MODEL_0de6786640ba4c66826f38f979ad7bae"
     }
    },
    "927dd36866c246d0b82c5aa91d8276cd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_de20adf09ea34f348e537cab9a6ad4a3",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_2b504abddd924dd0bfe9df0084a67779",
      "value": "model-00002-of-00004.safetensors:\u2007100%"
     }
    },
    "81dccaed1bc647609d5d423aebac4905": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d62c39ed5efa4eb69dd62249f0a34104",
      "max": 3996690997,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6394e11ce938451f8776884f15a1ea97",
      "value": 3996690997
     }
    },
    "da7d0349c2e54258950debfd5f895c6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_836e90e557a644b78f5e6aae5f0064fb",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_88b4b4002d5f44b9a50abaa09286dbd9",
      "value": "\u20074.00G/4.00G\u2007[00:26&lt;00:00,\u2007176MB/s]"
     }
    },
    "0de6786640ba4c66826f38f979ad7bae": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "de20adf09ea34f348e537cab9a6ad4a3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2b504abddd924dd0bfe9df0084a67779": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d62c39ed5efa4eb69dd62249f0a34104": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6394e11ce938451f8776884f15a1ea97": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "836e90e557a644b78f5e6aae5f0064fb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "88b4b4002d5f44b9a50abaa09286dbd9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f8889d1b3ed644ea8c3ac3a028a7068e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9bfb8a248810422eb1b1cf243ef5f70e",
       "IPY_MODEL_b5019e963aad46d7972f48c6ac40e359",
       "IPY_MODEL_2a462f6c77814d4689164e489cdafef6"
      ],
      "layout": "IPY_MODEL_880603aab32f499ea39ab8929580f549"
     }
    },
    "9bfb8a248810422eb1b1cf243ef5f70e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1cc1e0fe351f4d6eb895cc4fb1fdfac2",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_dfeed478a94742bfaf8dc3a4393506ad",
      "value": "model-00003-of-00004.safetensors:\u2007100%"
     }
    },
    "b5019e963aad46d7972f48c6ac40e359": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c95af8c087924fc5a80cff0466f215ca",
      "max": 3372033380,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_591b643077554d67afc157f6459df4f3",
      "value": 3372033380
     }
    },
    "2a462f6c77814d4689164e489cdafef6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c9b6dcec85b4427a25584574591343d",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_a695a855fe634145bbbb3f80b96e9f13",
      "value": "\u20073.37G/3.37G\u2007[00:23&lt;00:00,\u2007140MB/s]"
     }
    },
    "880603aab32f499ea39ab8929580f549": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1cc1e0fe351f4d6eb895cc4fb1fdfac2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dfeed478a94742bfaf8dc3a4393506ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c95af8c087924fc5a80cff0466f215ca": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "591b643077554d67afc157f6459df4f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0c9b6dcec85b4427a25584574591343d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a695a855fe634145bbbb3f80b96e9f13": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "80715c20d7f64aeba7fc93fa73ce57ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b518a7a211204d488c643839cecd6f2a",
       "IPY_MODEL_d52b9a58037e4aa59b80638b154f767f",
       "IPY_MODEL_4884f62654a84d0d9da407d07dce5395"
      ],
      "layout": "IPY_MODEL_95ffed1b5fda487fa8c44582286d7a02"
     }
    },
    "b518a7a211204d488c643839cecd6f2a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_73b5ef2bdd5f4530b3fe3c7ffaa87cbd",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_abfcd05c50ac46b6871a24448553a67b",
      "value": "model-00004-of-00004.safetensors:\u2007100%"
     }
    },
    "d52b9a58037e4aa59b80638b154f767f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d7493ba69c5b4aa59a17feaa329a2d93",
      "max": 1158267008,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7e26d06942354742b68d48ace780aada",
      "value": 1158267008
     }
    },
    "4884f62654a84d0d9da407d07dce5395": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_79029ed2c8ba4ccfb88a312205bc2560",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_dbbd29ed6e5a4989a03d9c9bde63763b",
      "value": "\u20071.16G/1.16G\u2007[00:12&lt;00:00,\u2007201MB/s]"
     }
    },
    "95ffed1b5fda487fa8c44582286d7a02": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "73b5ef2bdd5f4530b3fe3c7ffaa87cbd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "abfcd05c50ac46b6871a24448553a67b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d7493ba69c5b4aa59a17feaa329a2d93": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e26d06942354742b68d48ace780aada": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "79029ed2c8ba4ccfb88a312205bc2560": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dbbd29ed6e5a4989a03d9c9bde63763b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e0c520f8be5145aeb69f8d3cdfdefded": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5d58413bd64f4883b5fff2716a67d863",
       "IPY_MODEL_4b305b7d70084704b556a209f5cf0d16",
       "IPY_MODEL_bc744bbf637e42059d5d8696970245c3"
      ],
      "layout": "IPY_MODEL_1fcd9fbc82d74aefae254bced29034da"
     }
    },
    "5d58413bd64f4883b5fff2716a67d863": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e8c953a40bc641cfb44b015f430f2caf",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_4418d3e2b5684b009449ba9c4f5ceb85",
      "value": "Loading\u2007checkpoint\u2007shards:\u2007100%"
     }
    },
    "4b305b7d70084704b556a209f5cf0d16": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_124186dccf5947979e177079cf2ee5ce",
      "max": 4,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_15bbaf938b3e42c28f0f6a5321e278e3",
      "value": 4
     }
    },
    "bc744bbf637e42059d5d8696970245c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_da034ecc985f4afba2a18fd054396bee",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_ac2b080ba872426b8f16d9be72c1e457",
      "value": "\u20074/4\u2007[00:58&lt;00:00,\u200712.47s/it]"
     }
    },
    "1fcd9fbc82d74aefae254bced29034da": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e8c953a40bc641cfb44b015f430f2caf": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4418d3e2b5684b009449ba9c4f5ceb85": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "124186dccf5947979e177079cf2ee5ce": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "15bbaf938b3e42c28f0f6a5321e278e3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "da034ecc985f4afba2a18fd054396bee": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ac2b080ba872426b8f16d9be72c1e457": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "79299038cf414250bb952e889493f401": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_845801a6b02e489fb898fed0af364b0f",
       "IPY_MODEL_88b8b75214204c0bbf7e1f65ed178532",
       "IPY_MODEL_2e12a22d76434a54a9f84624c68b0827"
      ],
      "layout": "IPY_MODEL_09ba538e0f9c4c20a6b3d70f66189711"
     }
    },
    "845801a6b02e489fb898fed0af364b0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_913600deff7941648e6d603d234d4444",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_45aa84f7023e42d683989f2d7adfe597",
      "value": "generation_config.json:\u2007100%"
     }
    },
    "88b8b75214204c0bbf7e1f65ed178532": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_70cce90309354d1b92051f818fb296bf",
      "max": 165,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e1be78a2966d49ccb7587182258a6900",
      "value": 165
     }
    },
    "2e12a22d76434a54a9f84624c68b0827": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_95b897d89e534811af05fedb2d0096e9",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_2dfaafd2cd1549629c27596a2dbecf91",
      "value": "\u2007165/165\u2007[00:00&lt;00:00,\u200713.6kB/s]"
     }
    },
    "09ba538e0f9c4c20a6b3d70f66189711": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "913600deff7941648e6d603d234d4444": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "45aa84f7023e42d683989f2d7adfe597": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "70cce90309354d1b92051f818fb296bf": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e1be78a2966d49ccb7587182258a6900": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "95b897d89e534811af05fedb2d0096e9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2dfaafd2cd1549629c27596a2dbecf91": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "13bdb26dd860423bb09f6d1c4a11ab25": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_11acbc991f1c4b108f235ee780c08d37",
       "IPY_MODEL_8b102b75c3a44fb5a28e7d4d4e90387f",
       "IPY_MODEL_4df606171f4149b995c1ee5d123e72db"
      ],
      "layout": "IPY_MODEL_1c17d06dcf7c47db90a1b888453628e6"
     }
    },
    "11acbc991f1c4b108f235ee780c08d37": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_56a2b1b182c24fd188948ddb2a866c1d",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_392915e6341348e1a66d7ec80247ee46",
      "value": "tokenizer_config.json:\u2007"
     }
    },
    "8b102b75c3a44fb5a28e7d4d4e90387f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0525258525b2420d94afa5bb90c6a208",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_27e24a917d8a45c89f5908d4d1a8dd4f",
      "value": 1
     }
    },
    "4df606171f4149b995c1ee5d123e72db": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_31d30084c082437291a41b85c42cadd9",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_1198709f3ec947ec835267cd86509b4f",
      "value": "\u200722.8k/?\u2007[00:00&lt;00:00,\u20071.39MB/s]"
     }
    },
    "1c17d06dcf7c47db90a1b888453628e6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "56a2b1b182c24fd188948ddb2a866c1d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "392915e6341348e1a66d7ec80247ee46": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0525258525b2420d94afa5bb90c6a208": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "27e24a917d8a45c89f5908d4d1a8dd4f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "31d30084c082437291a41b85c42cadd9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1198709f3ec947ec835267cd86509b4f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "85ecf81b2616450a9351096b00c08a4a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_843fd9b6fafa43d383d080cd5a63584f",
       "IPY_MODEL_5605dddae8cd49bfaf1d036191c83bca",
       "IPY_MODEL_31c1f8f0865f4a678f9bf3a6b5bec4b7"
      ],
      "layout": "IPY_MODEL_5b39e9240f1b493b8b52107ffd7784ad"
     }
    },
    "843fd9b6fafa43d383d080cd5a63584f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8413f53dddb346b3ac896e4ba439c866",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_db9bca8fcb294c06ba9c17a1f54e93b3",
      "value": "tokenizer.json:\u2007100%"
     }
    },
    "5605dddae8cd49bfaf1d036191c83bca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_22dce123197a40cf9f33120115ec3f13",
      "max": 27868174,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_54b6936c6f7b4ef6a54c289547586d9c",
      "value": 27868174
     }
    },
    "31c1f8f0865f4a678f9bf3a6b5bec4b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a627360e9d9a416890add7a0d4478236",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_e30aba60806b40faa86c9a6530045f50",
      "value": "\u200727.9M/27.9M\u2007[00:00&lt;00:00,\u200748.4MB/s]"
     }
    },
    "5b39e9240f1b493b8b52107ffd7784ad": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8413f53dddb346b3ac896e4ba439c866": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db9bca8fcb294c06ba9c17a1f54e93b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "22dce123197a40cf9f33120115ec3f13": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "54b6936c6f7b4ef6a54c289547586d9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a627360e9d9a416890add7a0d4478236": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e30aba60806b40faa86c9a6530045f50": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3813c08be3a34e23a76dfefe05051d23": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4d8d1811fc44499dbc96d98645c1f2ca",
       "IPY_MODEL_50bfb4fa92d543cb914a2fb4a3dfcaf1",
       "IPY_MODEL_651fbe2f59d94b3782cde0e62a5329be"
      ],
      "layout": "IPY_MODEL_cbaf6db5c0884b0d8f78ca00c6ebafa3"
     }
    },
    "4d8d1811fc44499dbc96d98645c1f2ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_98320332cc0f43ba97ae9aa6b0ad2a31",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_f2ccc0686e27412c93e7e2bec5cc2ccb",
      "value": "special_tokens_map.json:\u2007100%"
     }
    },
    "50bfb4fa92d543cb914a2fb4a3dfcaf1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0126a176ba0a40e7a42ff644c9616a1d",
      "max": 446,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1aff4f81130649e6b2078616022365aa",
      "value": 446
     }
    },
    "651fbe2f59d94b3782cde0e62a5329be": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_26c68eb201604646985f4ae36b1956f8",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_794a7555aa504f8d889f2a048d4d0964",
      "value": "\u2007446/446\u2007[00:00&lt;00:00,\u200745.8kB/s]"
     }
    },
    "cbaf6db5c0884b0d8f78ca00c6ebafa3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "98320332cc0f43ba97ae9aa6b0ad2a31": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f2ccc0686e27412c93e7e2bec5cc2ccb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0126a176ba0a40e7a42ff644c9616a1d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1aff4f81130649e6b2078616022365aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "26c68eb201604646985f4ae36b1956f8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "794a7555aa504f8d889f2a048d4d0964": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "50bae2886c604003b94be2119244e23e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_802637e2aa744c1aa45cf6c828c5bfb2",
       "IPY_MODEL_863b12fb70f34ae8b809e0c2c36bb0bb",
       "IPY_MODEL_d39bff08794a416893cf475e23a129c4"
      ],
      "layout": "IPY_MODEL_e947ede222c24667b4b010408560e53a"
     }
    },
    "802637e2aa744c1aa45cf6c828c5bfb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b9245e1564534c08b9be1287975bbc8a",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_9fb1f6848ec343aebf17879448aca07f",
      "value": "chat_template.jinja:\u2007"
     }
    },
    "863b12fb70f34ae8b809e0c2c36bb0bb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fb814f046bbb46a481a1debf7abac837",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6cbed96579714c6aae308e8e6cddbc1e",
      "value": 1
     }
    },
    "d39bff08794a416893cf475e23a129c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_879d441943794e93b6be212fb2049e16",
      "placeholder": "\u200b",
      "style": "IPY_MODEL_5b7507d54185406f826992ee584f312c",
      "value": "\u200715.1k/?\u2007[00:00&lt;00:00,\u20071.69MB/s]"
     }
    },
    "e947ede222c24667b4b010408560e53a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b9245e1564534c08b9be1287975bbc8a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9fb1f6848ec343aebf17879448aca07f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fb814f046bbb46a481a1debf7abac837": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "6cbed96579714c6aae308e8e6cddbc1e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "879d441943794e93b6be212fb2049e16": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5b7507d54185406f826992ee584f312c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "state": {}
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}